Product image generation method, device, computer equipment and storage medium

Abstract

This application discloses a product image generation method, apparatus, computer device, and storage medium. The method includes: obtaining a first depth image corresponding to a calibration block at a first measurement perspective and a second depth image corresponding to a second measurement perspective, the first measurement perspective being different from the second measurement perspective; performing a first frame selection process on the first depth image according to the first measurement perspective to obtain a first detection frame, and performing a second frame selection process on the second depth image according to the second measurement perspective to obtain a second detection frame, the first frame selection process being different from the second frame selection process; and determining a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection frame and the point set included in the second detection frame. Using this application, higher measurement accuracy can be achieved in application scenarios where products are measured at multiple measurement perspectives.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment and storage medium for generating a product image. Background Art

[0002] With the development of product measurement technology, 3D measurement has been widely used in actual production to measure the properties of products. Among them, the properties can be 3D dimensions, flatness, etc.

[0003] In practical applications, calibration blocks and cameras are usually used together to complete 3D measurement of the product to be measured. Among them, the calibration block is a standard sample pre-designed according to the product to be measured. In 3D measurement, a reference coordinate system is established with the camera as a reference, and then the coordinate parameters of the camera for a given measurement position are used for measurement based on the reference coordinate system. Generally speaking, in 3D measurement, multiple acquisition cameras need to be arranged to acquire multiple depth images of the calibration block. Therefore, in the end, the depth images acquired by each acquisition camera need to be converted to a unified world coordinate system, so as to obtain the coordinate parameters of each measurement position in the world coordinate system, so as to determine the attribute characteristics of the product to be measured.

[0004] However, the 3D measurement methods in related technologies have low measurement accuracy and cannot be adapted to various 3D measurement projects. Summary of the Invention

[0005] In order to solve the above technical problems, the present application provides a product image generation method, device, computer equipment and storage medium, which can have higher measurement accuracy in application scenarios where products are measured from multiple measurement perspectives.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a product image, comprising:

[0007] Acquire a first depth image corresponding to the calibration block at a first measurement viewing angle and a second depth image corresponding to the calibration block at a second measurement viewing angle, respectively; the first measurement viewing angle is different from the second measurement viewing angle;

[0008] Performing a first frame selection process on the first depth image according to the first measurement perspective to obtain a first detection frame, and performing a second frame selection process on the second depth image according to the second measurement perspective to obtain a second detection frame; the first frame selection process and the second frame selection process are different;

[0009] Based on the point set included in the first detection box and the point set included in the second detection box, a point cloud measurement map of the product to be measured corresponding to the calibration block is determined.

[0010] In a second aspect, an embodiment of the present application provides a device for generating a product image, comprising:

[0011] An acquisition unit, configured to respectively acquire a first depth image corresponding to the calibration block at a first measurement viewing angle and a second depth image corresponding to the calibration block at a second measurement viewing angle; the first measurement viewing angle is different from the second measurement viewing angle;

[0012] a processing unit, configured to perform a first frame selection process on the first depth image according to a first measurement perspective to obtain a first detection frame, and perform a second frame selection process on the second depth image according to a second measurement perspective to obtain a second detection frame; the first frame selection process and the second frame selection process are different;

[0013] The determining unit is configured to determine a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection frame and the point set included in the second detection frame.

[0014] In a third aspect, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the product image generation method as described above is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the product image generation method as described above is implemented.

[0016] It can be seen that the embodiment of the present application provides a method for generating a product image, which first obtains a first depth image corresponding to the calibration block at a first measurement perspective and a second depth image corresponding to the second measurement perspective, respectively; wherein the first measurement perspective and the second measurement perspective are different. Since the calibration block has different feature representations at different measurement perspectives, different frame selection processes are selected for the depth images at different measurement perspectives, so that the detection frame corresponding to each measurement perspective matches the feature representation of the calibration block at that measurement perspective. Therefore, each detection frame can more accurately identify the area where the calibration block is located in each measurement perspective, which is beneficial to the subsequent measurement process. Based on this, it is possible to have higher measurement accuracy in application scenarios where products are measured at multiple measurement perspectives. Finally, the point set included in the first detection frame and the point set included in the second detection frame can be used to determine the point cloud measurement map of the product to be measured corresponding to the calibration block, so that the attribute characteristics of the product to be measured can be determined using the point cloud measurement map. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flowchart of a method for generating a product image provided in an embodiment of the present application;

[0019] Figure 2a A schematic diagram of a binarized image corresponding to a first depth image provided in an embodiment of the present application;

[0020] Figure 2b A schematic diagram of a first detection frame provided in an embodiment of the present application;

[0021] Figure 3a A schematic diagram of a binarized image corresponding to a second depth image provided in an embodiment of the present application;

[0022] Figure 3b A schematic diagram of a to-be-determined binarized image provided in an embodiment of the present application;

[0023] Figure 3c A schematic diagram of a second detection frame provided in an embodiment of the present application;

[0024] Figure 4a A schematic diagram of a top view of a calibration block provided in an embodiment of the present application;

[0025] Figure 4b A schematic diagram of a side view of a calibration block provided in an embodiment of the present application;

[0026] Figure 4c A schematic diagram of a partial view of a calibration component in a calibration block provided in an embodiment of the present application;

[0027] Figure 5a A schematic diagram of the layout of shooting positions provided in an embodiment of the present application;

[0028] Figure 5b A schematic diagram of an original depth image provided in an embodiment of the present application;

[0029] Figure 5c A schematic diagram of a full-view point cloud measurement map of a product provided in an embodiment of the present application;

[0030] Figure 6 A structural diagram of a product image generation device provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0034] In 3D measurement, calibration blocks are often used to measure the product to be measured. Typically, multiple measurement viewpoints are set up, and acquisition cameras are installed at each. These cameras can then capture depth images from different measurement viewpoints. Finally, by processing and performing coordinate conversion on these depth images from different measurement viewpoints, a point cloud measurement map of the product to be measured is generated, completing the 3D measurement. In practical applications, the point cloud measurement map can be used to determine the properties of the product to be measured, such as its 3D size and flatness. The point cloud measurement map can be an image composed of ordered points obtained by performing coordinate conversion on the depth image.

[0035] In practical applications, the calibration block exhibits different characteristic features under different measurement perspectives. For example, under the first measurement perspective, such as the top measurement perspective, the calibration block usually exhibits relatively regular features, while under the second measurement perspective, such as the inner and outer measurement perspectives, the calibration block usually exhibits less regular features. This is mainly because the surfaces of the calibration block can generally be divided into two categories: planes and convex surfaces. The plane surfaces are relatively regular, while the convex surfaces are irregular.

[0036] However, in the application scenario of measuring products under multiple measurement perspectives, the processing method in the relevant technology makes it possible that the detection frame determined for the depth images collected under different measurement perspectives cannot well conform to the feature performance under each measurement perspective. The detection frame determined in this way cannot accurately identify the area where the calibration block is located in each measurement perspective, which leads to the low accuracy of the point cloud measurement map obtained by subsequent processing of the depth image using the detection frame and coordinate conversion.

[0037] To this end, the present application provides a product image generation method, apparatus, computer equipment, and storage medium, which select different frame selection processes for depth images under different measurement perspectives, so that the detection frame corresponding to each measurement perspective matches the characteristic performance of the calibration block under the measurement perspective. Therefore, each detection frame can more accurately identify the area where the calibration block is located in each measurement perspective, thereby improving measurement accuracy.

[0038] The product image generation method provided in the embodiments of the present application can be implemented by a computer device, which can be a terminal device or a server; wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The terminal device and the server can be directly or indirectly connected via wired or wireless communication methods, and this application does not impose any restrictions on this.

[0039] Specifically described by the following examples:

[0040] Figure 1 This is a flowchart of a method for generating a product image provided in an embodiment of the present application. Taking a server as the aforementioned computer device as an example, the method includes:

[0041] S101: Acquire a first depth image corresponding to a calibration block at a first measurement viewing angle and a second depth image corresponding to a second measurement viewing angle, respectively.

[0042] In practical applications, for products to be measured, 3D measurement of the products to be measured is usually completed by constructing a calibration block. In one possible implementation method, the calibration block can be constructed in the following way: first, according to the morphological properties and measurement requirements of the product to be measured, multiple target calibration components can be determined from a number of calibration components to be determined, and then the multiple target calibration components are assembled to obtain a calibration block. Among them, the calibration components to be determined can be pre-produced standard samples; the morphological properties of the product to be measured are used to reflect the appearance shape of the product to be measured (such as a prism shape, etc.), and the measurement requirements are determined according to the business, such as only needing to measure 3D dimensions, or needing 3D dimensions and flatness, etc. Based on this, for different products to be measured, pre-produced standard samples can be directly used to flexibly assemble calibration blocks that are compatible with the current product to be measured, so that there is no need to produce corresponding calibration blocks separately for the current product to be measured, which can improve the reusability of calibration components and reduce costs.

[0043] In an application scenario where a product is measured at multiple measurement angles, a first and second measurement angles can be determined. The first and second measurement angles are different. The first measurement angle is the angle at which the calibration block's features appear relatively regular within the measurement angle, while the second measurement angle is the angle at which the calibration block's features appear irregular within the measurement angle. For example, the first measurement angle can be a top measurement angle, meaning the acquisition camera at the first measurement angle is located at the top of the calibration block. The second measurement angle can be an outside measurement angle, meaning the acquisition camera at the second measurement angle is located outside the calibration block.

[0044] It should be noted that this application does not impose any restrictions on the number of measurement angles included in the first measurement angle and the second measurement angle. For example, the second measurement angle may include not only the outer measurement angle but also the inner measurement angle. In another example, the second measurement angle may also include the inner arc angle measurement angle and the outer arc angle measurement angle. The number of measurement angles included in the first and second measurement angles and their number may be determined based on the current measurement requirements and the characteristic performance of the calibration block at each measurement angle.

[0045] In practical applications, a depth image is also called a range image. It refers to the distance between each point in the scene and the acquisition camera, and can be used as the Z coordinate of each point in the current camera coordinate system. The depth image can be calculated as point cloud data through coordinate transformation. Therefore, we can first obtain the first depth image corresponding to the first measurement perspective and the second depth image corresponding to the second measurement perspective of the calibration block. The first depth image can be acquired by a first acquisition camera installed at the first measurement perspective, and the second depth image can be acquired by a second acquisition camera installed at the second measurement perspective. The first acquisition camera and the second acquisition camera can be depth cameras, such as line scan depth cameras.

[0046] Based on this, the depth values ​​of each point in the first depth image are used to represent the distance between each surface point of the calibration block in the first measurement perspective and the first acquisition camera. The distance between each point and the first acquisition camera can be used as the Z coordinate of each point in the camera coordinate system of the first acquisition camera. The second depth image is similarly constructed. The first and second depth images can be used to subsequently determine the point cloud measurement map of the product to be measured.

[0047] S102: performing a first frame selection process on the first depth image according to the first measurement perspective to obtain a first detection frame, and performing a second frame selection process on the second depth image according to the second measurement perspective to obtain a second detection frame.

[0048] It can be understood that the first depth image and the second depth image are the original depth images collected by the first acquisition camera and the second acquisition camera respectively. In the actual acquisition process, since the acquisition camera is at a certain distance from the calibration block, that is, the field of view of the acquisition camera includes not only the calibration block but also a part of the area near the calibration block. Therefore, it is necessary to determine a detection frame from the original depth image to identify the area where the calibration block is located. In a specific implementation, a first frame selection process can be performed on the first depth image according to the first measurement perspective to obtain a first detection frame, and a second frame selection process can be performed on the second depth image according to the second measurement perspective to obtain a second detection frame. The first frame selection process is different from the second frame selection process. Based on this, different frame selection processes can be selected for depth images under different measurement perspectives, so that the detection frame under each measurement perspective matches the characteristic performance of the calibration block under the measurement perspective. Therefore, each detection frame can more accurately represent the area where the calibration block is located in each measurement perspective.

[0049] In the original depth image, the pixel value of each point is within a certain range, and usually the pixel values ​​of each point are different. In practical applications, the range of the pixel value distribution of each point in the original depth image is mainly related to the acquisition camera model and the storage method of the depth image. Different storage methods correspond to different ranges. Common storage methods include Uint16, Int16, Clout32, Double, etc. For example, the range corresponding to Int16 is [-2 -15 ,2 15 -1], and the interval range of Clout32 is [-2 -31 ,2 31-1]. In practical applications, the corresponding acquisition camera model and storage method can be flexibly selected according to the actual acquisition requirements (such as acquisition accuracy requirements). In the binary image, the pixel value of each point is only 0 or 255, which is simpler and easier to determine the detection frame than the original depth image. Therefore, in order to facilitate the determination of the detection frame, in a possible implementation method, the first depth image can be first binarized according to the first measurement perspective to obtain a binary image corresponding to the first depth image. Since the calibration block features under the first measurement perspective are relatively regular, the first detection frame can be directly determined according to the pixel value of each point in the binary image corresponding to the first depth image. For the second depth image, the second depth image can first be binarized according to the second measurement angle of view to obtain a binarized image corresponding to the second depth image. Since the calibration block features at the second measurement angle of view are irregular, concave region detection can be performed on the binarized image corresponding to the second depth image to obtain a target concave region. The target concave region in the binarized image corresponding to the second depth image is then removed to obtain a pending binarized image. Finally, the second detection frame is determined based on the pixel values ​​of each point in the pending binarized image. Based on this, the target concave region present in the binarized image due to the irregular calibration block features can be removed, thereby avoiding interference of the target concave region in determining the second detection frame and improving the accuracy of the second detection frame.

[0050] It should be noted that this application does not impose any restrictions on the specific implementation of the binarization process. For example, the use of a thresholded binarization process specifically refers to determining a binarization threshold based on the original depth image. Points in the original depth image whose pixel values ​​are greater than or equal to the binarization threshold can be considered as valid areas, so the pixel values ​​of these points are converted to 255; points in the original depth image whose pixel values ​​are less than the binarization threshold can be considered as invalid areas, so the pixel values ​​of these points are converted to 0, thereby achieving binarization of the depth image. In an embodiment of the present application, the valid area can be the area where the index block is located, and the invalid area can be other areas outside the index block that appear in the field of view of the acquisition camera.

[0051] Based on the idea of ​​binarization processing, in a possible implementation method, removing the target concave area in the binarized image corresponding to the second depth image to obtain the pending binarized image may include: performing pixel value inversion processing on the pixel values ​​of the points in the target concave area to obtain the pending binarized image; since the pixel value of the point in the target concave area in the binarized image is 255, but the target concave area is actually an invalid area, the pixel value of the point in the target concave area can be converted from 255 to 0 in the pixel value inversion processing. After completion, the pending binarized image can be obtained.

[0052] It can be understood that a continuous distribution of multiple points with the same pixel value can form an area. Accordingly, the area formed by points with a pixel value of 0 is an invalid area, and the area formed by points with a pixel value of 255 is a valid area. In one possible implementation, when determining the first detection frame based on the pixel values ​​of each point in the binarized image corresponding to the first depth image, you can first use the points with target pixel values ​​in the binarized image corresponding to the first depth image to generate multiple first areas to be determined; wherein the target pixel value is the 255 pixel value in the binarization process. For example, taking the first measurement perspective as the top measurement perspective as an example, the binarized image corresponding to the first depth image can be seen in Figure 2a As shown; the white area is the first pending area, and the pixel value of the midpoint in this area is 255; the black area is the invalid area, and the pixel value of the midpoint in this area is 0. In actual applications, there may also be noise points, etc., which may cause some interference areas to exist in the first pending area. Generally speaking, the area of ​​the interference area is smaller than the area of ​​the actual valid area. Therefore, it is also possible to filter the target first pending area from multiple first pending areas based on the area sizes of the multiple first pending areas, such as selecting a rectangular area with the area Top-k as the target first pending area. Based on this, the target first pending area obtained after filtering out the interference area is the final valid area, and finally the first detection frame can be determined according to the area boundary of the target first pending area.

[0053] Similarly, when determining the second detection frame based on the pixel values ​​of each point in the pending binary image, multiple second pending regions can be generated using the points in the pending binary image whose pixel values ​​are the target pixel values. For example, taking the second measurement angle as the outer measurement angle as an example, the binary image corresponding to the second depth image can be seen as follows: Figure 3a As shown, the undetermined binary image obtained after performing concave area detection on the binary image and removing the target concave area can be seen in Figure 3b As shown in the figure, the white area is the second pending area, and the pixel value of the midpoint in this area is 255; the black area is the invalid area, and the pixel value of the midpoint in this area is 0. Then, based on the size of the multiple second pending areas, the target second pending area can be filtered from the multiple second pending areas. For example, a rectangular area with an area of ​​Top-r can be selected as the target second pending area. Based on this, the target second pending area obtained after removing the interference area is the final valid area. Finally, the second detection frame can be determined based on the area boundary of the target second pending area.

[0054] It should be noted that this application does not impose any restrictions on the settings of k and r. For example, for a calibration block assembled from multiple target calibration components, k and r can be set according to the number of target calibration components that appear in the calibration block under a specific measurement perspective.

[0055] It should also be noted that this application does not limit the specific implementation of determining the first detection frame based on the area boundary of the first target area to be determined, and determining the second detection frame based on the area boundary of the second target area to be determined. For ease of understanding, this application provides the following two determination methods as examples:

[0056] Because the first target area to be determined and the second target area to be determined are used to represent the actual valid area, in one possible implementation, the boundary of the first target area to be determined can be directly used as the first detection frame, and the boundary of the second target area to be determined can be used as the second detection frame. Based on this, the detection frames under each measurement perspective can be quickly determined.

[0057] In another possible implementation, it is also possible to appropriately expand the area boundaries of the first target area to be determined and the second target area to be determined, so that the area range becomes larger to ensure that the calibration block is completely contained in the detection frame. In a specific implementation, the area boundaries of the first target area to be determined can be expanded, and the expanded area boundaries are used as the first detection frame. Similarly, the area boundaries of the second target area to be determined are expanded, and the expanded area boundaries are used as the second detection frame. Based on this, the area identified by the first detection frame is larger than the first target area to be determined, and the area identified by the second detection frame is larger than the second target area to be determined, so that the calibration block is completely contained in the detection frame.

[0058] In practical applications, measurement engineers can determine the amplitude of the expansion processing based on empirical values, while avoiding excessive expansion and extending the invalid area into the detection frame.

[0059] It should be noted that the number of the first detection frame and the second detection frame is not limited in this application. Figure 2a For example, the first detection frame obtained after expansion can be seen in Figure 2b As shown, it includes three first detection frames. Figure 3b For example, the second detection frame obtained after expansion processing can be seen in Figure 3c As shown, a second detection frame is included. That is, three detection frames are determined by measuring the viewing angle at the top, and one detection frame is determined by measuring the viewing angle at the outside. That is, in this embodiment, k=3 and r=1.

[0060] S103: Determine a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection frame and the point set included in the second detection frame.

[0061] The point set included in the first detection frame is used to represent the surface points of the calibration block at the first measurement perspective, and the point set included in the second detection frame is used to represent the surface points of the calibration block at the second measurement perspective.

[0062] In practical applications, plane detection can be performed on the point set included in the first detection frame to obtain M first fitting planes, and plane detection can be performed on the point set included in the second detection frame to obtain M second fitting planes, where M is a positive integer. Corner point calculation can then be performed using the M first fitting planes to obtain N first corner points, and corner point calculation can be performed using the M second fitting planes to obtain N second corner points, where N is a positive integer. The N first corner points obtained in this way can be considered as the vertices of the calibration block at the first measurement perspective, and the N second corner points can be considered as the vertices of the calibration block at the second measurement perspective. Finally, the N first corner points and N second corner points can be used to determine the point cloud measurement map of the product to be measured corresponding to the calibration block.

[0063] It should be noted that this application does not limit the specific implementation method of plane detection using the point set included in the detection box. For ease of understanding, this application embodiment provides a two-stage plane detection algorithm as an example:

[0064] In the first stage, plane detection can be performed on the point set included in the first detection frame to obtain M first detection planes. In specific implementation, for example, the RASAC plane detection algorithm can be used to detect multiple first planes formed by the point set included in the first detection frame. When there are too few points remaining in the point set included in the first detection frame or when no planes are detected, the detection can be stopped, and then the plane with the top M areas can be selected from the multiple first planes according to the area size as the first detection plane. Similarly, plane detection can be performed on the point set included in the second detection frame to obtain M second detection planes.

[0065] In the second stage, an erosion operation is performed on each of the M first detection planes obtained in the first stage to obtain M first undetermined planes. Based on this, the plane boundaries and surrounding outliers are removed by the erosion operation to achieve a denoising and smoothing effect. As a result, the first undetermined plane has a higher accuracy than the first detection plane. In a specific implementation, an erosion operation can be first performed on each first detection plane to obtain a corresponding mask. The plane formed by the point set included in the mask is the corresponding first undetermined plane. In order to further improve the accuracy, the point set included in each of the M first undetermined planes can be used for fitting processing to obtain M first fitting planes. In other words, the point set in each first undetermined plane is refitted to obtain the corresponding first fitting plane. Similarly, an erosion operation can be performed on each of the M second detection planes to obtain M second undetermined planes, and then a fitting process can be performed on the point set included in each of the M second undetermined planes to obtain M second fitting planes. In the fitting process, the average vertical distance between each point in the undetermined plane and its corresponding fitting plane is minimized. The fitting plane obtained based on this has higher flatness, which is conducive to improving the subsequent measurement accuracy.

[0066] It should be noted that this application does not impose any limitation on the settings of M and N. For example, M=6 and N=8 can be set according to actual measurement requirements and the morphological properties of the product to be measured.

[0067] It is understood that the first corner point can be considered as the vertex of the calibration block at the first measurement angle, and the second corner point can be considered as the vertex of the calibration block at the second measurement angle, and the vertex is the intersection of any three surfaces in the calibration block. For ease of understanding, this embodiment of the application takes M=6 and N=8 as an example to introduce the specific implementation method of using M fitting planes to calculate the corner points to obtain N corner points:

[0068] First, the cosine values ​​can be calculated using the normal vectors of any two of the six fitting planes, and the two fitting planes with the largest cosine values ​​are taken as the top and bottom surfaces. Based on this, the fitting planes that are more likely to be the top and bottom surfaces can be screened out from the six fitting planes. For the remaining four fitting planes, they can be sorted clockwise or counterclockwise according to the coordinates of the center points of each fitting plane. Based on this, the six fitting planes can form a cube. Furthermore, the intersection points of the three fitting planes can be obtained in sequence to obtain eight ordered corner points. Among them, the coordinates of the center point of the fitting plane refer to the coordinates in the camera coordinate system of the corresponding acquisition camera.

[0069] It can be understood that, for the process of using M first fitting planes to calculate the corner points to obtain N first corner points, and using M second fitting planes to calculate the corner points to obtain N second corner points, reference can be made to the above-mentioned method of obtaining the corner points, which will not be described in detail here.

[0070] It is understood that different measurement perspectives have different camera coordinate systems. To enable a holistic assessment, it is necessary to convert each point from the camera coordinate systems at different measurement perspectives into a unified world coordinate system. This allows for the determination of a point cloud measurement map of the product to be measured through coordinate transformation. In a specific implementation, the camera coordinates of the first corner point in the first camera coordinate system and the camera coordinates of the second corner point in the second camera coordinate system can be obtained. The first camera coordinate system is the camera coordinate system of the first acquisition camera installed at the first measurement perspective, and the second camera coordinate system is the camera coordinate system of the second acquisition camera installed at the second measurement perspective. Furthermore, an affine transformation matrix is ​​calculated for the first corner point, the second corner point, and the multiple vertices of the calibration block in the world coordinate system. Finally, the point cloud measurement map of the product to be measured can be determined based on the affine transformation matrix and the multiple depth images obtained of the product to be measured. The world coordinate system is established with the calibration block as a reference. For example, the world coordinate system can be established with the center of the calibration block as the origin and the axes parallel to the long side and wide side of the calibration block as the y-axis and x-axis, respectively.

[0071] It should be noted that, in the world coordinate system, the three-dimensional coordinates of each vertex of the calibration block are known. For example, the three-dimensional coordinates of each vertex can be determined through a CAD three-dimensional drawing of the calibration block.

[0072] It can be seen that the embodiment of the present application provides a method for generating a product image, which first obtains a first depth image corresponding to the calibration block at a first measurement perspective and a second depth image corresponding to the second measurement perspective, respectively; wherein the first measurement perspective and the second measurement perspective are different. Since the calibration block has different feature representations at different measurement perspectives, different frame selection processes are selected for the depth images at different measurement perspectives, so that the detection frame corresponding to each measurement perspective matches the feature representation of the calibration block at that measurement perspective. Therefore, each detection frame can more accurately identify the area where the calibration block is located in each measurement perspective, which is beneficial to the subsequent measurement process. Based on this, it is possible to have higher measurement accuracy in application scenarios where products are measured at multiple measurement perspectives. Finally, the point set included in the first detection frame and the point set included in the second detection frame can be used to determine the point cloud measurement map of the product to be measured corresponding to the calibration block, so that the attribute characteristics of the product to be measured can be determined using the point cloud measurement map.

[0073] For ease of understanding, the present embodiment of the application uses a calibration component to construct a calibration block corresponding to the product A to be measured. For details, see Figure 4aAs shown, a top view of the calibration block is shown, which can be seen in Figure 4b As shown, a side view of the calibration block is shown, which can be seen in Figure 4c As shown, a partial view of the calibration component in the calibration block is shown, and the viewing angle of the partial view is the viewing angle corresponding to the side view of the calibration block.

[0074] Corresponding to Figure 4a According to the actual measurement requirements, it is determined that for the current product A to be measured, full-angle measurement must be completed under five measurement angles: inner measurement angle, outer measurement angle, top measurement angle, inner arc angle measurement angle, and outer arc angle measurement angle. Among them, the top measurement angle is the first measurement angle mentioned above, and the inner measurement angle, outer measurement angle, inner arc angle measurement angle, and outer arc angle measurement angle are the second measurement angles mentioned above. Since only a part can be captured at one time, the following settings are set: Figure 5a The shooting positions shown include a total of 20 shooting positions, namely, four shooting positions under the outer measurement perspective, namely outer (1), outer (2), outer (3) and outer (4), four shooting positions under the inner measurement perspective, namely inner (1), inner (2), inner (3) and inner (4), four shooting positions under the top measurement perspective, namely top (1), top (2), top (3) and top (4), four shooting positions under the inner arc angle measurement perspective, namely inner arc (1), inner arc (2), inner arc (3) and inner arc (4), and four shooting positions under the outer arc angle measurement perspective, namely outer arc (1), outer arc (2), outer arc (3) and outer arc (4).

[0075] Since the inner measurement angle and the inner arc angle measurement angle are distributed on the same side, similarly, the outer measurement angle and the outer arc angle measurement angle are distributed on the same side. Therefore, in practical applications, three acquisition cameras can be installed, which are responsible for capturing images of the top measurement angle, the inner measurement angle and the inner arc angle measurement angle, and the outer measurement angle and the outer arc angle measurement angle, respectively. It is understandable that after the three acquisition cameras are installed, their relative positions remain unchanged. In practical applications, the entire acquisition device can be mounted on a motion platform to facilitate the control of the acquisition camera's movement around the axis. Based on this, a part can be shot at a time, and after shooting, the movement can be controlled to shoot other parts, ultimately achieving shooting of all 20 shooting positions. Among them, the motion platform can be a high-precision five-axis motion platform that can control the acquisition device to move around the X-axis, Y-axis, and Z-axis, so that the acquisition camera can move to different shooting positions for shooting.

[0076] Corresponding to the installation method of this acquisition camera, during actual shooting, the above 20 shooting positions can be divided into eight groups. The first group can include outer (1), inner (1), and top (1); the second group can include outer (2), inner (2), and top (2); the third group can include outer (3), inner (3), and top (3); the fourth group can include outer (4), inner (4), and top (4); the fifth group can include outer arc (1) and inner arc (1); the sixth group can include outer arc (2) and inner arc (2); the seventh group can include outer arc (3) and inner arc (3); the eighth group can include outer arc (4) and inner arc (4). For details, please refer to Figure 5a Based on this, a total of 20 original depth images can be obtained through eight shots. Correspondingly, the 20 original depth images can be found in Figure 5b shown.

[0077] Furthermore, the binary images corresponding to the 20 original depth images can be processed using the method provided in the embodiment of the present application, and finally converted to a unified world coordinate system to obtain a complete product full-view image of the product A to be measured. It should be noted that through coordinate transformation, the camera coordinate system can be converted to the world coordinate system. In the world coordinate system, the product full-view image is the point cloud measurement image. For details, please refer to Figure 5c It should be noted that in actual applications, the point cloud measurement map can be set as a color map for higher differentiation.

[0078] In practical applications, we can use Figure 5c The point cloud measurement image of the product from all angles shown can determine the 3D size, flatness and other attribute characteristics of the product A to be measured.

[0079] It can be understood that it basically corresponds to the method embodiment, so relevant points can refer to the partial description of the method embodiment.

[0080] Figure 6 A structural diagram of a product image generation device provided in an embodiment of the present application includes:

[0081] An acquisition unit 601 is configured to acquire a first depth image corresponding to a calibration block at a first measurement viewing angle and a second depth image corresponding to a second measurement viewing angle, respectively; the first measurement viewing angle is different from the second measurement viewing angle;

[0082] a processing unit 602 configured to perform a first frame selection process on the first depth image according to a first measurement perspective to obtain a first detection frame, and perform a second frame selection process on the second depth image according to a second measurement perspective to obtain a second detection frame; the first frame selection process and the second frame selection process are different;

[0083] The determining unit 603 is configured to determine a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection frame and the point set included in the second detection frame.

[0084] In some embodiments, in performing a first frame selection process on the first depth image according to the first measurement perspective to obtain a first detection frame, the processing unit 602 is specifically configured to:

[0085] Binarize the first depth image according to the first measurement perspective to obtain a binarized image corresponding to the first depth image;

[0086] Determine a first detection frame according to the pixel value of each point in the binarized image corresponding to the first depth image;

[0087] In performing the second frame selection process on the second depth image according to the second measurement perspective to obtain the second detection frame, the processing unit 602 is specifically configured to:

[0088] performing binarization processing on the second depth image according to the second measurement viewing angle to obtain a binarized image corresponding to the second depth image;

[0089] Performing concave region detection on the binary image corresponding to the second depth image to obtain a target concave region;

[0090] removing the target concave area in the binarized image corresponding to the second depth image to obtain a pending binarized image;

[0091] A second detection frame is determined according to the pixel value of each point in the to-be-determined binarized image.

[0092] In some embodiments, in determining the first detection frame according to the pixel value of each point in the binarized image corresponding to the first depth image, the processing unit 602 is specifically configured to:

[0093] Generate a plurality of first undetermined areas using points whose pixel values ​​are target pixel values ​​in the binarized image corresponding to the first depth image;

[0094] screening a target first to-be-determined area from the plurality of first to-be-determined areas according to the sizes of the plurality of first to-be-determined areas;

[0095] Determine a first detection frame according to the region boundary of the first target area to be determined;

[0096] In terms of determining the second detection frame according to the pixel value of each point in the to-be-determined binarized image, the processing unit 602 is specifically configured to:

[0097] Generate a plurality of second areas to be determined using points whose pixel values ​​in the binary image to be determined are target pixel values;

[0098] screening a target second to-be-determined area from the plurality of second to-be-determined areas according to the sizes of the plurality of second to-be-determined areas;

[0099] A second detection frame is determined according to the area boundary of the second undetermined area of ​​the target.

[0100] In some embodiments, in determining the first detection frame according to the area boundary of the first target area to be determined, the processing unit 602 is specifically configured to:

[0101] Expanding the boundary of the first undetermined area of ​​the target to obtain a first detection frame;

[0102] In determining the second detection frame according to the area boundary of the second target area to be determined, the processing unit 602 is specifically configured to:

[0103] The boundary of the second target area to be determined is expanded to obtain a second detection frame.

[0104] In some embodiments, in terms of removing the target concave area in the binarized image corresponding to the second depth image to obtain the undetermined binarized image, the processing unit 602 is specifically configured to:

[0105] The pixel values ​​of the points in the target concave area are inverted to obtain a binary image to be determined.

[0106] In some embodiments, in determining a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection box and the point set included in the second detection box, the determining unit 603 is specifically configured to:

[0107] Perform plane detection on the point set included in the first detection frame to obtain M first fitting planes, and perform plane detection on the point set included in the second detection frame to obtain M second fitting planes; M is a positive integer;

[0108] Using M first fitting planes to calculate corner points, N first corner points are obtained; using M second fitting planes to calculate corner points, N second corner points are obtained; N is a positive integer;

[0109] The N first corner points and the N second corner points are used to determine a point cloud measurement map of the product to be measured corresponding to the calibration block.

[0110] In some embodiments, in performing plane detection on the point set included in the first detection frame to obtain M first fitting planes, the determining unit 603 is specifically configured to:

[0111] Performing plane detection on the point set included in the first detection frame to obtain M first detection planes;

[0112] Performing an erosion operation on each of the M first detection planes to obtain M first undetermined planes;

[0113] Performing fitting processing using the point set included in each of the M first undetermined planes to obtain M first fitting planes;

[0114] In terms of performing plane detection on the point set included in the second detection frame to obtain M second fitting planes, the determining unit 603 is specifically configured to:

[0115] Perform plane detection on the point set included in the second detection frame to obtain M second detection planes;

[0116] Performing an erosion operation on each of the M second detection planes to obtain M second undetermined planes;

[0117] The point set included in each of the M second undetermined planes is used to perform fitting processing to obtain M second fitting planes.

[0118] In some embodiments, in determining a point cloud measurement map of the product to be measured corresponding to the calibration block using the N first corner points and the N second corner points, the determining unit 603 is specifically configured to:

[0119] Obtain the camera coordinates of the first corner point in a first camera coordinate system and the camera coordinates of the second corner point in a second camera coordinate system, respectively; the first camera coordinate system is the camera coordinate system of a first acquisition camera installed at a first measurement viewing angle, and the second camera coordinate system is the camera coordinate system of a second acquisition camera installed at a second measurement viewing angle;

[0120] Calculating affine transformation matrices for the first corner point, the second corner point, and multiple vertices of the calibration block in the world coordinate system; the world coordinate system is established with the calibration block as a reference;

[0121] A point cloud measurement map of the product to be measured is determined based on the affine transformation matrix and the acquired multiple depth images of the product to be measured.

[0122] In some embodiments, the calibration block is constructed as follows:

[0123] Determine multiple target calibration components from a number of pending calibration components according to the morphological properties and measurement requirements of the product to be measured;

[0124] A calibration block is obtained by assembling multiple target calibration components.

[0125] It can be seen that the embodiment of the present application provides a method for generating a product image, which first obtains a first depth image corresponding to the calibration block at a first measurement perspective and a second depth image corresponding to the second measurement perspective, respectively; wherein the first measurement perspective and the second measurement perspective are different. Since the calibration block has different feature representations at different measurement perspectives, different frame selection processes are selected for the depth images at different measurement perspectives, so that the detection frame corresponding to each measurement perspective matches the feature representation of the calibration block at that measurement perspective. Therefore, each detection frame can more accurately identify the area where the calibration block is located in each measurement perspective, which is beneficial to the subsequent measurement process. Based on this, it is possible to have higher measurement accuracy in application scenarios where products are measured at multiple measurement perspectives. Finally, the point set included in the first detection frame and the point set included in the second detection frame can be used to determine the point cloud measurement map of the product to be measured corresponding to the calibration block, so that the attribute characteristics of the product to be measured can be determined using the point cloud measurement map.

[0126] The present application embodiment provides a computer device, the structural diagram of which can be seen in Figure 7 The computer device 700 includes a processor 701 and a memory 702. The memory 702 stores a computer program. When the processor 701 executes the computer program, the product image generation method provided in the above embodiment is implemented.

[0127] The computer device may include a terminal device or a server, and the aforementioned product image generating device may be configured in the computer device.

[0128] The present application provides a computer-readable storage medium, the structure of which can be seen in Figure 8 The computer-readable storage medium 800 stores a computer program, and when the computer program is executed by the processor, the product image generation method provided in the above embodiment is implemented.

[0129] It will be understood by those skilled in the art that all or part of the steps for implementing the above method embodiments can be accomplished by hardware related to computer program instructions, and the computer program can be stored in a computer-readable storage medium 800. This application does not impose any restrictions on the form of the computer-readable storage medium 800. For example, the computer-readable storage medium 800 can be at least one of the following storage media: a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other storage media that can store computer program codes. For example, the random access memory can be a static random access memory (SRAM) or a dynamic random access memory (DRAM).

[0130] It should be noted that, for the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0131] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further restrictions, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the above elements.

[0132] The above describes in detail the product image generation method, apparatus, computer device, and storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is intended only to facilitate understanding of the methods of the present application. Furthermore, those skilled in the art will appreciate that variations in the specific implementation methods and scope of application of the methods of the present application may occur.

[0133] In summary, the contents of this specification should not be construed as limiting this application. Any changes or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Moreover, based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations.

Claims

1. A method for generating a product image, characterized in that: include: Acquire a first depth image corresponding to the calibration block at a first measurement viewing angle and a second depth image corresponding to the calibration block at a second measurement viewing angle respectively; The first measurement viewing angle is different from the second measurement viewing angle, and the feature of the calibration block at the first measurement viewing angle is a plane, while the feature of the calibration block at the second measurement viewing angle is a convex surface; performing a first frame selection process on the first depth image according to the first measurement perspective to obtain a first detection frame, and performing a second frame selection process on the second depth image according to the second measurement perspective to obtain a second detection frame; the first frame selection process and the second frame selection process are different; Determining a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection frame and the point set included in the second detection frame; The performing a first frame selection process on the first depth image according to the first measurement perspective to obtain a first detection frame includes: performing binarization processing on the first depth image according to the first measurement viewing angle to obtain a binarized image corresponding to the first depth image; determining a first detection frame according to a pixel value of each point in the binarized image corresponding to the first depth image; The performing a second frame selection process on the second depth image according to the second measurement perspective to obtain a second detection frame includes: performing binarization processing on the second depth image according to the second measurement viewing angle to obtain a binarized image corresponding to the second depth image; Performing concave region detection on the binary image corresponding to the second depth image to obtain a target concave region; removing the target concave area in the binarized image corresponding to the second depth image to obtain a pending binarized image; A second detection frame is determined according to the pixel value of each point in the to-be-determined binarized image.

2. The method according to claim 1, characterized in that The determining of the first detection frame according to the pixel value of each point in the binarized image corresponding to the first depth image includes: Generate a plurality of first to-be-determined areas using points whose pixel values ​​are target pixel values ​​in the binarized image corresponding to the first depth image; screening a target first to-be-determined area from the plurality of first to-be-determined areas according to the sizes of the plurality of first to-be-determined areas; Determine a first detection frame according to the region boundary of the first target region to be determined; The determining of the second detection frame according to the pixel value of each point in the to-be-determined binarized image includes: Generate a plurality of second areas to be determined using points whose pixel values ​​in the binary image to be determined are the target pixel values; screening a target second to-be-determined area from the plurality of second to-be-determined areas according to the sizes of the plurality of second to-be-determined areas; A second detection frame is determined according to the area boundary of the second target area to be determined.

3. The method according to claim 2, characterized in that The determining of the first detection frame according to the region boundary of the first target region to be determined includes: Expanding the boundary of the first undetermined area of ​​the target to obtain a first detection frame; The determining of the second detection frame according to the area boundary of the second target area to be determined includes: The boundary of the second target area to be determined is expanded to obtain a second detection frame.

4. The method according to claim 1, wherein The removing the target concave area in the binarized image corresponding to the second depth image to obtain a pending binarized image includes: The pixel values ​​of the midpoints in the target concave area are subjected to pixel value inversion processing to obtain a pending binary image.

5. The method according to any one of claims 1 to 4, characterized in that The step of determining a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection frame and the point set included in the second detection frame includes: Performing plane detection on the point set included in the first detection frame to obtain M first fitting planes, and performing plane detection on the point set included in the second detection frame to obtain M second fitting planes; where M is a positive integer; Using the M first fitting planes to perform corner point calculation to obtain N first corner points, and using the M second fitting planes to perform corner point calculation to obtain N second corner points, where N is a positive integer; A point cloud measurement map of the product to be measured corresponding to the calibration block is determined using the N first corner points and the N second corner points.

6. The method according to claim 5, characterized in that The performing plane detection on the point set included in the first detection frame to obtain M first fitting planes includes: Performing plane detection on the point set included in the first detection frame to obtain M first detection planes; Performing an erosion operation on each of the M first detection planes to obtain M first undetermined planes; Performing fitting processing using the point set included in each of the M first undetermined planes to obtain M first fitting planes; The performing plane detection on the point set included in the second detection frame to obtain M second fitting planes includes: Performing plane detection on the point set included in the second detection frame to obtain M second detection planes; Performing an erosion operation on each of the M second detection planes to obtain M second undetermined planes; Fitting processing is performed using the point set included in each of the M second planes to be determined to obtain M second fitting planes.

7. The method according to claim 5, characterized in that The method of determining a point cloud measurement image of the product to be measured corresponding to the calibration block by using the N first corner points and the N second corner points includes: Obtain the camera coordinates of the first corner point in a first camera coordinate system and the camera coordinates of the second corner point in a second camera coordinate system, respectively; the first camera coordinate system is the camera coordinate system of a first acquisition camera installed at the first measurement angle of view, and the second camera coordinate system is the camera coordinate system of a second acquisition camera installed at the second measurement angle of view; Calculating affine transformation matrices for the first corner point, the second corner point, and multiple vertices of the calibration block in a world coordinate system; the world coordinate system is established with the calibration block as a reference; A point cloud measurement map of the product to be measured is determined based on the affine transformation matrix and the acquired multiple depth images of the product to be measured.

8. The method according to claim 1, characterized in that The calibration block is constructed as follows: Determining a plurality of target calibration components from a plurality of pending calibration components according to the morphological properties and measurement requirements of the product to be measured; A plurality of the target calibration components are assembled to obtain the calibration block.

9. A product image generating device, characterized in that: include: An acquisition unit, configured to respectively acquire a first depth image corresponding to the calibration block at a first measurement viewing angle and a second depth image corresponding to the calibration block at a second measurement viewing angle; The first measurement viewing angle is different from the second measurement viewing angle, and the feature of the calibration block at the first measurement viewing angle is a plane, while the feature of the calibration block at the second measurement viewing angle is a convex surface; a processing unit, configured to perform a first frame selection process on the first depth image according to the first measurement perspective to obtain a first detection frame, and perform a second frame selection process on the second depth image according to the second measurement perspective to obtain a second detection frame; the first frame selection process and the second frame selection process are different; a determining unit, configured to determine a point cloud measurement map of the product to be measured corresponding to the calibration block based on the point set included in the first detection frame and the point set included in the second detection frame; The processing unit is specifically configured to: performing binarization processing on the first depth image according to the first measurement viewing angle to obtain a binarized image corresponding to the first depth image; determining a first detection frame according to a pixel value of each point in the binarized image corresponding to the first depth image; performing binarization processing on the second depth image according to the second measurement viewing angle to obtain a binarized image corresponding to the second depth image; Performing concave region detection on the binary image corresponding to the second depth image to obtain a target concave region; removing the target concave area in the binarized image corresponding to the second depth image to obtain a pending binarized image; A second detection frame is determined according to the pixel value of each point in the to-be-determined binarized image.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images