Point cloud region-of-interest division method and device, computer equipment and medium

Abstract

The invention relates to a point cloud region-of-interest division method and device, computer equipment and a medium, and the method comprises the steps: obtaining a template image of an original image, and generating a mask image of the template image; calculating the similarity between local areas in the template image and the original image, and searching an ROI initial area based on the similarity; according to the mask image, obtaining pixel coordinates of the ROI initial region; and generating a point cloud ROI division result according to the pixel coordinates. In the whole process, the initial position of the ROI is determined through template matching based on similarity, pixel-by-pixel processing is carried out on the matching result through the mask image, and an accurate ROI division result can be obtained.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud data processing, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for dividing a point cloud region of interest. Background Art

[0002] In the process of inspection or measurement, the area that needs attention is usually not all pixels in the camera's field of view, but only a part of the surface of the inspected workpiece, called ROI (Region of Interest). If it is not processed, the whole field of view is directly reconstructed and the point cloud data obtained is inspected, the result will be interfered by non-ROI, and the inspection efficiency will also be reduced. Therefore, it is particularly important to quickly and effectively divide the measurement results of ROI.

[0003] Traditional mainstream ROI segmentation methods can be divided into two categories. The first method is to calibrate a fixed point cloud ROI in advance. The specific steps are to set the coordinate range of the ROI in space, so as to clarify its boundaries in the x, y, and z directions of the template point cloud. Then, the points that meet these ranges are screened out in the three-dimensional point cloud obtained by actual measurement, and these points are retained to achieve the ROI segmentation process. The second method is to divide the ROI based on point cloud registration technology. This method first requires manually dividing a piece of point cloud ROI data, and then using this data and the measured point cloud data to align it using an algorithm. Through this registration, the ROI area can be approximately overlapped with certain areas of the measured point cloud, thereby extracting the overlapping area and finally achieving ROI segmentation.

[0004] Although the above-mentioned traditional ROI division method can realize ROI division, its division accuracy is easily affected by various factors (such as changes in the measured physical position, rough surface of the measured object, poor imaging quality, etc.), and the overall ROI division accuracy is low, and accurate ROI division cannot be achieved. Summary of the invention

[0005] Based on this, it is necessary to provide an accurate point cloud region of interest division method, device, computer equipment, storage medium and computer program product to address the above technical problems.

[0006] In a first aspect, the present application provides a method for dividing a point cloud region of interest. The method comprises:

[0007] Acquire a template image of the original image and generate a mask image of the template image;

[0008] Calculating the similarity between the template image and the local area in the original image, and finding the ROI initial area based on the similarity;

[0009] According to the mask image, the pixel coordinates of the initial area of ​​the ROI are obtained;

[0010] A point cloud ROI division result is generated according to the pixel coordinates.

[0011] In one embodiment, the acquiring a template image of the original image and generating a mask image of the template image comprises:

[0012] Get the template image of the original image;

[0013] Draw a two-dimensional bounding box of the ROI area using a rectangular frame in the template image;

[0014] The precise position of the ROI area is selected pixel by pixel in the bounding box, and a mask image of the ROI is obtained.

[0015] In one embodiment, the mask image of ROI is:

[0016]

[0017] Where T(i, j) is the gray value of the template image.

[0018] In one embodiment, the calculating the similarity between the template image and the local area in the original image, and finding the ROI initial area based on the similarity includes:

[0019] Selecting a matching method, wherein the matching method includes at least one of square difference matching, normalized square difference matching, and correlation coefficient matching;

[0020] Sliding the template image on the original image based on a preset sliding window, and applying the selected matching method at each position to calculate a similarity score with the template image to form a correlation graph;

[0021] Find the point with the highest value in the correlation graph to determine the matching position;

[0022] According to the matching position, an initial ROI area is determined.

[0023] In one embodiment, the matching method includes normalized square difference matching; the normalized matching correlation coefficient is:

[0024]

[0025] Where (i, j) is the coordinate on the measured image, (m, n) is the coordinate of the template image, S is the grayscale of the original image, T is the grayscale of the template image, and M and N are the length and width of the original image, respectively.

[0026] In one embodiment, generating a point cloud ROI division result according to the pixel coordinates includes:

[0027] Constructing a coordinate matrix, wherein the coordinate matrix has the same size as the original image, each matrix point corresponds to a pixel of the original image one-to-one, and each matrix point stores a Z coordinate or an XYZ coordinate;

[0028] According to the pixel coordinates, the Z value of the corresponding point in the coordinate matrix is ​​reset to an invalid value;

[0029] The reset coordinate matrix is ​​traversed to generate a point cloud ROI division result.

[0030] In one embodiment, traversing the reset coordinate matrix to generate a point cloud ROI division result includes:

[0031] Traversing the reset coordinate matrix;

[0032] If the Z value of the current point in the reset coordinate matrix is ​​an invalid value, skip it;

[0033] The points whose Z values ​​are valid are stored in the same point cloud to obtain the point cloud ROI division result.

[0034] In a second aspect, the present application also provides a point cloud region of interest division device. The device comprises:

[0035] An image acquisition module, used to acquire a template image of an original image and generate a mask image of the template image;

[0036] A similarity processing module, used to calculate the similarity between the template image and the local area in the original image, and find the ROI initial area based on the similarity;

[0037] A pixel coordinate acquisition module, used to obtain the pixel coordinates of the initial area of ​​the ROI according to the mask image;

[0038] The division module is used to generate a point cloud ROI division result according to the pixel coordinates.

[0039] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0040] Acquire a template image of the original image and generate a mask image of the template image;

[0041] Calculating the similarity between the template image and the local area in the original image, and finding the ROI initial area based on the similarity;

[0042] According to the mask image, the pixel coordinates of the initial area of ​​the ROI are obtained;

[0043] A point cloud ROI division result is generated according to the pixel coordinates.

[0044] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Acquire a template image of the original image and generate a mask image of the template image;

[0046] Calculating the similarity between the template image and the local area in the original image, and finding the ROI initial area based on the similarity;

[0047] According to the mask image, the pixel coordinates of the initial area of ​​the ROI are obtained;

[0048] A point cloud ROI division result is generated according to the pixel coordinates.

[0049] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0050] Acquire a template image of the original image and generate a mask image of the template image;

[0051] Calculating the similarity between the template image and the local area in the original image, and finding the ROI initial area based on the similarity;

[0052] According to the mask image, the pixel coordinates of the initial area of ​​the ROI are obtained;

[0053] A point cloud ROI division result is generated according to the pixel coordinates.

[0054] The above-mentioned point cloud region of interest division method, device, computer equipment, storage medium and computer program product obtain a template image of the original image, generate a mask image of the template image; calculate the similarity between the template image and the local area in the original image, and find the ROI initial area based on the similarity; obtain the pixel coordinates of the ROI initial area according to the mask image; and generate the point cloud ROI division result according to the pixel coordinates. In the whole process, the initial position of the ROI is determined by template matching based on similarity, and the matching result is processed pixel by pixel through the mask image to obtain an accurate ROI division result. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1A diagram of an application environment of a method for dividing a point cloud region of interest in one embodiment;

[0056] Figure 2 It is a schematic diagram of a flow chart of a method for dividing a point cloud region of interest in one embodiment;

[0057] Figure 3 A schematic diagram of a flow chart of a method for dividing a point cloud region of interest in another embodiment;

[0058] Figure 4 A schematic diagram of the 2D to 3D mapping process in an application example;

[0059] Figure 5 It is a structural block diagram of a device for dividing a point cloud region of interest in one embodiment;

[0060] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] The point cloud region of interest division method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 sends a point cloud region of interest division request to the server 104, and the server 104 responds to the request, obtains the template image of the original image, and generates a mask image of the template image; calculates the similarity between the template image and the local area in the original image, and finds the ROI initial area based on the similarity; according to the mask image, the pixel coordinates of the ROI initial area are obtained; according to the pixel coordinates, the point cloud ROI division result is generated. Further, the server 104 can feed back the point cloud ROI division result to the terminal 102. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0063] In one embodiment, Figure 2As shown in FIG. 1 , a method for dividing a point cloud region of interest is provided, and the method is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0064] S200: Acquire a template image of the original image and generate a mask image of the template image.

[0065] The original image refers to the image taken of the device under test. The template image refers to the template made based on the original image. After the template image is obtained, the template image is further processed to generate a mask image of the template image.

[0066] Specifically, a template image is an image that represents or features a target or object of interest. In practical applications, this template image may be obtained through preprocessing, selection, or design to highlight specific areas or objects of interest to the user or application. A mask image is a binary image in which the pixel value of the region of interest (ROI) is 1 (or white) and the pixel value of the region of no interest is 0 (or black). This mask image is used to accurately extract the pixel coordinates of the ROI in subsequent steps.

[0067] S400: Calculate the similarity between the template image and the local area in the original image, and find the ROI initial area based on the similarity.

[0068] Similarity calculation involves comparing the template image with each local area in the original image to determine which areas are most similar to the template image. Commonly used similarity calculation methods include correlation coefficient, mean square error (MSE), structural similarity (SSIM), etc. The ROI initial region is based on the similarity calculation results, and the local area with the highest similarity is selected as the initial estimate of the ROI. The purpose of this step is to narrow the search range and provide a more accurate starting point for subsequent steps.

[0069] S600: Obtain pixel coordinates of the initial ROI region according to the mask image.

[0070] Using the mask image generated earlier, extract all pixel coordinates belonging to the ROI from the ROI initial area. This step is the key to connecting the image processing field and the point cloud processing field, because the pixel coordinates will be used for the subsequent point cloud ROI division.

[0071] S800: Generate a point cloud ROI division result according to the pixel coordinates.

[0072] Map the extracted pixel coordinates to the corresponding point cloud data. This usually involves converting the image coordinates into point cloud coordinates in 3D space, which may require additional camera calibration and depth information. Based on the mapped point cloud coordinates, determine the ROI in the point cloud. This step may involve cropping, segmenting, or marking the point cloud to highlight the ROI.

[0073] The above point cloud region of interest division method obtains the template image of the original image and generates the mask image of the template image; calculates the similarity between the template image and the local area in the original image, and finds the ROI initial area based on the similarity; obtains the pixel coordinates of the ROI initial area according to the mask image; and generates the point cloud ROI division result according to the pixel coordinates. In the whole process, the initial position of the ROI is determined by template matching based on similarity, and then the matching result is processed pixel by pixel through the mask image to obtain an accurate ROI division result.

[0074] like Figure 3 As shown, in one embodiment, S200 includes:

[0075] S220: Acquire a template image of the original image.

[0076] Here, an image representing or characterizing the target or object of interest is obtained, that is, a template image. This image can be cropped from the original image or generated by other means (such as image processing algorithms).

[0077] S240: Draw a two-dimensional bounding box of the ROI region in the template image using a rectangular frame.

[0078] Use a rectangular box to manually or automatically draw a two-dimensional bounding box of the region of interest (ROI) on the template image. This bounding box should tightly surround the ROI so that the subsequent steps can accurately extract the mask map of the ROI, use image processing algorithms (such as edge detection, threshold segmentation, etc.) to identify the boundaries of the ROI, and finally draw a two-dimensional bounding box of the ROI area.

[0079] S260: Select the exact position of the ROI area pixel by pixel in the bounding box, and obtain a mask image of the ROI.

[0080] Based on the bounding box drawn in the previous step, check the pixels within the bounding box pixel by pixel to determine which pixels belong to the ROI. This usually involves some form of classification or judgment of each pixel within the bounding box to determine whether it belongs to the ROI. Once the precise location of the ROI is determined, the mask map of the ROI can be generated. The mask map is a binary image of the same size as the template image, where the pixel value within the ROI area is 1 (or white), and the pixel value outside the ROI area is 0 (or black). In this way, the mask map accurately represents the position and shape of the ROI in the template image. More specifically, the mask map of the ROI is:

[0081]

[0082] Where T(i, j) is the gray value of the template image.

[0083] In one embodiment, calculating the similarity between the template image and the local region in the original image, and finding the ROI initial region based on the similarity includes:

[0084] Step 1: Select a matching method, which includes at least one of square difference matching, normalized square difference matching, and correlation coefficient matching.

[0085] In this step, you need to select one or more of the preset matching methods to calculate the similarity between the template image and the local area in the original image. These matching methods include square difference matching, normalized square difference matching, and correlation coefficient matching. Square difference matching calculates the sum of the squares of the pixel value differences between the template image and the local area in the original image. The smaller the value, the more similar it is. Normalized square difference matching normalizes the square difference to eliminate the influence of different brightness or contrast on the matching results. Correlation coefficient matching calculates the correlation coefficient between the template image and the local area in the original image. The closer the value is to 1, the more similar it is.

[0086] Step 2: Slide the template image on the original image based on the preset sliding window, and apply the selected matching method at each position to calculate the similarity score with the template image to form a correlation graph.

[0087] In this step, a preset sliding window (usually the same size as the template image or slightly larger) is used to slide the template image on the original image. Each step of the sliding window represents a new position of the template image on the original image. For the template image at each position of the sliding window, the selected matching method is applied to calculate the similarity score between it and the local area in the original image. These scores will form a correlation graph, where each point represents a position of the template image on the original image and its corresponding similarity score.

[0088] Step 3: Find the point with the highest value in the correlation graph and determine the matching position.

[0089] In this step, find the point with the highest similarity score in the correlation graph. This point represents the best matching position of the template image on the original image, that is, the best matching position of the initial region of the ROI.

[0090] Step 4: Determine the initial ROI area based on the matching position.

[0091] Finally, based on the determined matching position, the ROI initial region can be further determined, which usually involves defining a rectangular region as the ROI initial region based on the size of the template image or a preset ROI size, with the matching position as the center.

[0092] In this embodiment, a specific method is provided to calculate the similarity between the template image and the local area in the original image, and find the ROI initial area based on the similarity. By selecting a suitable matching method, sliding the template image on the original image using a sliding window, calculating the similarity score and forming a correlation graph, finding the point with the highest value, and determining the matching position and the ROI initial area, the method can accurately locate the ROI initial area in the original image.

[0093] Specifically, the matching method includes normalized square difference matching; the normalized matching correlation coefficient is:

[0094]

[0095] Where (i, j) is the coordinate on the measured image, (m, n) is the coordinate of the template image, S is the grayscale of the original image, T is the grayscale of the template image, and M and N are the length and width of the original image, respectively.

[0096] In one embodiment, generating a point cloud ROI division result according to pixel coordinates includes:

[0097] Step 1: Construct a coordinate matrix. The size of the coordinate matrix is ​​the same as that of the original image, and each matrix point corresponds to a pixel of the original image one by one. Each matrix point stores a Z coordinate or an XYZ coordinate.

[0098] In this step, first construct a coordinate matrix with the same size as the original image. Each point of this matrix corresponds one-to-one to each pixel of the original image. Each point in the coordinate matrix can store the Z coordinate (if there is only depth information) or the XYZ coordinate (if it contains complete three-dimensional information). This matrix is ​​actually a representation of three-dimensional data, where the X and Y coordinates are implicitly given by the row and column indices of the matrix, while the Z coordinate (or the Z component in the XYZ coordinate) is explicitly stored in the matrix. Furthermore, the file format of the coordinate matrix can be a csv, depth map, ordered point cloud ply or pcd file that corresponds one-to-one to the pixels of the original image.

[0099] Step 2: According to the pixel coordinates, reset the Z value of the corresponding point in the coordinate matrix to an invalid value.

[0100] In this step, the pixel coordinates obtained previously (these coordinates represent the position of the ROI in the original image) are used to find the corresponding points in the coordinate matrix. Then, the Z values ​​of these corresponding points (or the Z component in the XYZ coordinates) are reset to an invalid value. This invalid value can be a specific value or a special marker value used to identify these points in subsequent processing. For example, it can be set to an invalid value NAN to complete the mapping from 2D to 3D. The specific setting implementation is as follows Figure 4 shown.

[0101] Step 3: Traverse the reset coordinate matrix to generate the point cloud ROI division result.

[0102] In this step, the entire reset coordinate matrix is ​​traversed. For each point, check whether its Z value (or the Z component in the XYZ coordinates) is an invalid value set previously. If the Z value is an invalid value, the point is excluded from the point cloud ROI division result. If the Z value is a valid value, the point is included in the point cloud ROI division result. Finally, by traversing the entire matrix and applying the above rules, a point cloud ROI division result containing only the points inside the ROI can be generated.

[0103] In one embodiment, traversing the reset coordinate matrix to generate the point cloud ROI division result includes: traversing the reset coordinate matrix; if the Z value of the current point in the reset coordinate matrix is ​​an invalid value, skipping; storing the points with valid Z values ​​in the same point cloud to obtain the point cloud ROI division result. The overall 2D to 3D mapping and the final point cloud ROI division result are as follows: Figure 4 shown.

[0104] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0105] Based on the same inventive concept, the embodiment of the present application also provides a point cloud region of interest division device for implementing the above-mentioned point cloud region of interest division method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more point cloud region of interest division device embodiments provided below can refer to the above-mentioned limitations on the point cloud region of interest division method, and will not be repeated here.

[0106] In one embodiment, Figure 5 As shown, a point cloud region of interest division device is provided, comprising:

[0107] The image acquisition module 200 is used to acquire a template image of the original image and generate a mask image of the template image;

[0108] A similarity processing module 400 is used to calculate the similarity between the template image and the local area in the original image, and find the ROI initial area based on the similarity;

[0109] The pixel coordinate acquisition module 600 is used to obtain the pixel coordinates of the initial area of ​​the ROI according to the mask image;

[0110] The division module 800 is used to generate a point cloud ROI division result according to pixel coordinates.

[0111] In one embodiment, the image acquisition module 200 is also used to acquire a template image of the original image; draw a two-dimensional bounding box of the ROI area using a rectangular frame in the template image; select the precise position of the ROI area pixel by pixel in the bounding box, and acquire a mask image of the ROI.

[0112] In one embodiment, the mask image of ROI is:

[0113]

[0114] Where T(i, j) is the gray value of the template image.

[0115] In one embodiment, the similarity processing module 400 is also used to select a matching method, which includes at least one of square difference matching, normalized square difference matching, and correlation coefficient matching; sliding the template image on the original image based on a preset sliding window, and applying the selected matching method at each position to calculate the similarity score with the template image to form a correlation graph; finding the point with the highest value in the correlation graph to determine the matching position; and determining the ROI initial area based on the matching position.

[0116] In one embodiment, the matching method includes normalized square difference matching; the normalized matching correlation coefficient is:

[0117]

[0118] Where (i, j) is the coordinate on the measured image, (m, n) is the coordinate of the template image, S is the grayscale of the original image, T is the grayscale of the template image, and M and N are the length and width of the original image, respectively.

[0119] In one embodiment, the division module 800 is also used to construct a coordinate matrix, the size of the coordinate matrix is ​​the same as that of the original image, and each matrix point corresponds one-to-one to a pixel of the original image, and each matrix point stores a Z coordinate or an XYZ coordinate; according to the pixel coordinate, the Z value of the corresponding point in the coordinate matrix is ​​reset to an invalid value; the reset coordinate matrix is ​​traversed to generate a point cloud ROI division result.

[0120] In one embodiment, the division module 800 is also used to traverse the reset coordinate matrix; if the Z value of the current point in the reset coordinate matrix is ​​an invalid value, it is skipped; the points with valid Z values ​​are stored in the same point cloud to obtain the point cloud ROI division result.

[0121] Each module in the above-mentioned point cloud region of interest division device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0122] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store preset data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for dividing a point cloud region of interest is implemented.

[0123] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0125] Obtain a template image of the original image and generate a mask image of the template image;

[0126] Calculate the similarity between the template image and the local area in the original image, and find the initial ROI area based on the similarity;

[0127] According to the mask image, the pixel coordinates of the initial area of ​​ROI are obtained;

[0128] Generate point cloud ROI division results based on pixel coordinates.

[0129] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0130] Obtain a template image of the original image; use a rectangular frame to draw a two-dimensional bounding box of the ROI area in the template image; select the exact position of the ROI area pixel by pixel in the bounding box, and obtain a mask image of the ROI.

[0131] In one embodiment, the mask image of ROI is:

[0132]

[0133] Where T(i, j) is the gray value of the template image.

[0134] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0135] Selecting a matching method, the matching method including at least one of square difference matching, normalized square difference matching, and correlation coefficient matching; sliding the template image on the original image based on a preset sliding window, and applying the selected matching method at each position to calculate the similarity score with the template image to form a correlation graph; finding the point with the highest value in the correlation graph to determine the matching position;

[0136] According to the matching position, the initial ROI area is determined.

[0137] In one embodiment, the matching method includes normalized square difference matching; the normalized matching correlation coefficient is:

[0138]

[0139] Where (i, j) is the coordinate on the measured image, (m, n) is the coordinate of the template image, S is the grayscale of the original image, T is the grayscale of the template image, and M and N are the length and width of the original image, respectively.

[0140] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0141] Construct a coordinate matrix whose size is the same as the original image, and each matrix point corresponds to the original image pixel one by one. Each matrix point stores the Z coordinate or XYZ coordinate. According to the pixel coordinate, reset the Z value of the corresponding point in the coordinate matrix to an invalid value. Traverse the reset coordinate matrix to generate the point cloud ROI division result.

[0142] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0143] Traverse the reset coordinate matrix; if the Z value of the current point in the reset coordinate matrix is ​​an invalid value, skip it; store the points with valid Z values ​​in the same point cloud to obtain the point cloud ROI division result.

[0144] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0145] Obtain a template image of the original image and generate a mask image of the template image;

[0146] Calculate the similarity between the template image and the local area in the original image, and find the initial ROI area based on the similarity;

[0147] According to the mask image, the pixel coordinates of the initial area of ​​ROI are obtained;

[0148] Generate point cloud ROI division results based on pixel coordinates.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] Obtain a template image of the original image; use a rectangular frame to draw a two-dimensional bounding box of the ROI area in the template image; select the exact position of the ROI area pixel by pixel in the bounding box, and obtain a mask image of the ROI.

[0151] In one embodiment, the mask image of ROI is:

[0152]

[0153] Where T(i, j) is the gray value of the template image.

[0154] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0155] Selecting a matching method, the matching method including at least one of square difference matching, normalized square difference matching, and correlation coefficient matching; sliding the template image on the original image based on a preset sliding window, and applying the selected matching method at each position to calculate the similarity score with the template image to form a correlation graph; finding the point with the highest value in the correlation graph to determine the matching position;

[0156] According to the matching position, the initial ROI area is determined.

[0157] In one embodiment, the matching method includes normalized square difference matching; the normalized matching correlation coefficient is:

[0158]

[0159] Where (i, j) is the coordinate on the measured image, (m, n) is the coordinate of the template image, S is the grayscale of the original image, T is the grayscale of the template image, and M and N are the length and width of the original image, respectively.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0161] Construct a coordinate matrix whose size is the same as the original image, and each matrix point corresponds to the original image pixel one by one. Each matrix point stores the Z coordinate or XYZ coordinate. According to the pixel coordinate, reset the Z value of the corresponding point in the coordinate matrix to an invalid value. Traverse the reset coordinate matrix to generate the point cloud ROI division result.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] Traverse the reset coordinate matrix; if the Z value of the current point in the reset coordinate matrix is ​​an invalid value, skip it; store the points with valid Z values ​​in the same point cloud to obtain the point cloud ROI division result.

[0164] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0165] Obtain a template image of the original image and generate a mask image of the template image;

[0166] Calculate the similarity between the template image and the local area in the original image, and find the initial ROI area based on the similarity;

[0167] According to the mask image, the pixel coordinates of the initial area of ​​ROI are obtained;

[0168] Generate point cloud ROI division results based on pixel coordinates.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0170] Obtain a template image of the original image; use a rectangular frame to draw a two-dimensional bounding box of the ROI area in the template image; select the exact position of the ROI area pixel by pixel in the bounding box, and obtain a mask image of the ROI.

[0171] In one embodiment, the mask image of ROI is:

[0172]

[0173] Where T(i, j) is the gray value of the template image.

[0174] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0175] Selecting a matching method, the matching method including at least one of square difference matching, normalized square difference matching, and correlation coefficient matching; sliding the template image on the original image based on a preset sliding window, and applying the selected matching method at each position to calculate the similarity score with the template image to form a correlation graph; finding the point with the highest value in the correlation graph to determine the matching position;

[0176] According to the matching position, the initial ROI area is determined.

[0177] In one embodiment, the matching method includes normalized square difference matching; the normalized matching correlation coefficient is:

[0178]

[0179] Where (i, j) is the coordinate on the measured image, (m, n) is the coordinate of the template image, S is the grayscale of the original image, T is the grayscale of the template image, and M and N are the length and width of the original image, respectively.

[0180] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0181] Construct a coordinate matrix whose size is the same as the original image, and each matrix point corresponds to the original image pixel one by one. Each matrix point stores the Z coordinate or XYZ coordinate. According to the pixel coordinate, reset the Z value of the corresponding point in the coordinate matrix to an invalid value. Traverse the reset coordinate matrix to generate the point cloud ROI division result.

[0182] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0183] Traverse the reset coordinate matrix; if the Z value of the current point in the reset coordinate matrix is ​​an invalid value, skip it; store the points with valid Z values ​​in the same point cloud to obtain the point cloud ROI division result.

[0184] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0185] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0186] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for dividing a point cloud region of interest, characterized in that: The method comprises: Acquire a template image of the original image and generate a mask image of the template image; Calculating the similarity between the template image and the local area in the original image, and finding the ROI initial area based on the similarity; According to the mask image, the pixel coordinates of the initial area of ​​the ROI are obtained; A point cloud ROI division result is generated according to the pixel coordinates.

2. The method according to claim 1, characterized in that The step of acquiring a template image of an original image and generating a mask image of the template image comprises: Get the template image of the original image; Draw a two-dimensional bounding box of the ROI area using a rectangular frame in the template image; The precise position of the ROI area is selected pixel by pixel in the bounding box, and a mask image of the ROI is obtained.

3. The method according to claim 2, characterized in that The mask image of ROI is: Where T(i, j) is the gray value of the template image.

4. The method according to claim 1, characterized in that: The calculating the similarity between the template image and the local area in the original image, and finding the ROI initial area based on the similarity includes: Selecting a matching method, wherein the matching method includes at least one of square difference matching, normalized square difference matching, and correlation coefficient matching; Sliding the template image on the original image based on a preset sliding window, and applying the selected matching method at each position to calculate a similarity score with the template image to form a correlation graph; Find the point with the highest value in the correlation graph to determine the matching position; According to the matching position, an initial ROI area is determined.

5. The method according to claim 4, characterized in that The matching method includes normalized square difference matching; the normalized matching correlation coefficient is: Where (i, j) is the coordinate on the measured image, (m, n) is the coordinate of the template image, S is the grayscale of the original image, T is the grayscale of the template image, and M and N are the length and width of the original image, respectively.

6. The method according to claim 1, characterized in that Generating a point cloud ROI division result according to the pixel coordinates includes: Constructing a coordinate matrix, wherein the coordinate matrix has the same size as the original image, each matrix point corresponds to a pixel of the original image one-to-one, and each matrix point stores a Z coordinate or an XYZ coordinate; According to the pixel coordinates, the Z value of the corresponding point in the coordinate matrix is ​​reset to an invalid value; The reset coordinate matrix is ​​traversed to generate a point cloud ROI division result.

7. The method according to claim 6, characterized in that The traversing the reset coordinate matrix to generate a point cloud ROI division result includes: Traversing the reset coordinate matrix; If the Z value of the current point in the reset coordinate matrix is ​​an invalid value, skip it; The points whose Z values ​​are valid are stored in the same point cloud to obtain the point cloud ROI division result.

8. A point cloud region of interest division device, characterized in that: The device comprises: An image acquisition module, used to acquire a template image of an original image and generate a mask image of the template image; A similarity processing module, used to calculate the similarity between the template image and the local area in the original image, and find the ROI initial area based on the similarity; A pixel coordinate acquisition module, used to obtain the pixel coordinates of the initial area of ​​the ROI according to the mask image; The division module is used to generate a point cloud ROI division result according to the pixel coordinates.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images