Equipment detection method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a device detection method, an electronic device and a computer readable storage medium, and the method comprises the steps: obtaining a first three-dimensional image of a to-be-detected device, and segmenting a plurality of two-dimensional images in the first three-dimensional image, so as to obtain a target region image in the plurality of two-dimensional images; obtaining a second three-dimensional image of the to-be-detected equipment based on a target area image in the plurality of two-dimensional images; modeling the to-be-detected equipment based on the first three-dimensional image to obtain a point cloud model of the to-be-detected equipment, and registering the point cloud model with the second three-dimensional image to obtain a detection image of the to-be-detected equipment. According to the invention, the detection efficiency and detection precision of the equipment can be improved, and standardization and large-scale detection of the equipment can be realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a device detection method, an electronic device and a computer-readable storage medium. Background Art

[0002] In the current industrial manufacturing field, equipment inspection mainly relies on traditional manual visual inspection methods. This method usually involves technicians using the naked eye or auxiliary tools to carefully inspect the surface and edges of the mold to identify any possible defects or wear. In addition, some automated inspection technologies have also begun to be used in equipment inspection, such as machine vision systems for 2D image processing, which analyze the digital image of the mold and use algorithms such as edge detection and threshold segmentation to identify the contour and surface features of the mold. However, manual inspection is not only time-consuming but also easily affected by subjective judgment, making it difficult to ensure the accuracy of the inspection results. Although automated 2D image processing technologies have increased the inspection speed, they are difficult to meet the requirements of high-precision inspection when dealing with equipment with complex shapes, complex background noise, or equipment under complex ambient lighting conditions. In short, although manual inspection methods are intuitive, they are inefficient, easily affected by personal experience and fatigue, and difficult to achieve standardization and scale. Although automated 2D image processing systems have increased speed, they are limited by the limitations of their planar vision and are difficult to accurately capture the three-dimensional features and complex surface details of the mold. Summary of the invention

[0003] The purpose of this application is to provide a device detection method, an electronic device, and a computer-readable storage medium, which can improve the detection efficiency and detection accuracy of the device and realize the standardization and scale of device detection.

[0004] To achieve the above objectives:

[0005] In a first aspect, an embodiment of the present application provides a device detection method, the method comprising:

[0006] Acquire a first three-dimensional image of the device to be detected, and segment a plurality of two-dimensional images in the first three-dimensional image to obtain target area images in the plurality of two-dimensional images;

[0007] Based on the target area image in the multiple two-dimensional images, obtaining a second three-dimensional image of the device to be detected;

[0008] The device to be detected is modeled based on the first three-dimensional image to obtain a point cloud model of the device to be detected, and the point cloud model is registered with the second three-dimensional image to obtain a detection image of the device to be detected.

[0009] In one embodiment, the first three-dimensional image has a first three-dimensional coordinate system formed according to pixel points;

[0010] The step of segmenting the plurality of two-dimensional images in the first three-dimensional image to obtain target area images in the plurality of two-dimensional images includes:

[0011] Obtaining two-dimensional images corresponding to a plurality of surfaces of the first three-dimensional image according to the two-dimensional coordinates in the first three-dimensional coordinate system;

[0012] Performing plane correction on the two-dimensional images corresponding to the multiple surfaces to obtain the best fitting plane for each surface;

[0013] The best fitting plane is segmented to obtain the target area image.

[0014] In one embodiment, the segmenting based on the best fitting plane to obtain the target area image includes:

[0015] identifying a target region in the plurality of best-fit planes;

[0016] The target area is segmented according to the two-dimensional coordinates of the target area to obtain target area images in the multiple best fitting planes.

[0017] In one embodiment, obtaining a second three-dimensional image of the device to be detected based on the target area image in the multiple two-dimensional images includes:

[0018] The target area images in the multiple best fitting planes are combined and output according to the relative positions of the multiple best fitting planes in the first three-dimensional image to obtain a second three-dimensional image of the device to be detected.

[0019] In one embodiment, before registering the point cloud data with the second three-dimensional image to obtain the detection result of the device to be detected, the method further includes:

[0020] Identify local concave parts and local convex parts of the target area image in the multiple two-dimensional images, calculate the distance between any first target point in the second three-dimensional image and the convex hull boundary closest to it, so as to determine whether the first target point is located in the local concave part according to the distance;

[0021] When the distance is greater than a preset distance threshold, determining that the first target point is in a local concave portion;

[0022] When it is determined that the first target point is located in a local concave portion, the first target point is separated to distinguish different characteristic regions of the device to be detected, thereby obtaining a segmented second three-dimensional image.

[0023] In one embodiment, registering the point cloud data with the second three-dimensional image to obtain a detection result of the device to be detected includes:

[0024] Acquire a second target point corresponding to the first target point in the second three-dimensional image from the point cloud model, and acquire a rotation matrix that minimizes the distance between the first target point and the second target point;

[0025] Iteratively calculating the rotation matrix so that the rotation matrix reaches a preset convergence standard;

[0026] When the rotation matrix reaches a preset convergence standard, the corresponding second three-dimensional image is determined as the detection image of the device to be detected.

[0027] In one embodiment, the iterative calculation of the transformation matrix so that the transformation matrix reaches a preset convergence standard includes:

[0028] Iteratively updating the rotation matrix and the translation vector to reduce the error between the first target point and the second target point;

[0029] The error is calculated by the following formula:

[0030]

[0031] Where J is the error between the first target point and the second target point, p i represents the first target point, q i represents the second target point, i represents the point cloud sequence number in the point cloud model and the second three-dimensional image, n represents the total number of point clouds in the point cloud model, R is the rotation matrix, and t is the translation vector, which represents the translation required to align the second target point with the third target point.

[0032] In one embodiment, the step of acquiring a first three-dimensional image of the device to be detected includes:

[0033] Use a three-dimensional scanner to scan the device to be detected at multiple angles to obtain corresponding scanning results;

[0034] The scanning results are merged to obtain the three-dimensional coordinates of each point on the surface of the device to be detected;

[0035] The three-dimensional image is obtained based on the three-dimensional coordinates.

[0036] In a second aspect, the present application provides an electronic device, including: a processor;

[0037] a memory for storing instructions executable by the processor;

[0038] The processor is configured to execute the instructions to implement the device detection method as described in any one of the first aspects.

[0039] In a third aspect, the present application provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor, the device detection method as described in any one of the first aspects is implemented.

[0040] The device detection method, electronic device and computer-readable storage medium provided by the present application include: obtaining a first three-dimensional image of the device to be detected, segmenting multiple two-dimensional images in the first three-dimensional image, and obtaining a target area image in the multiple two-dimensional images; obtaining a second three-dimensional image of the device to be detected based on the target area image in the multiple two-dimensional images; modeling the device to be detected based on the first three-dimensional image, obtaining a point cloud model of the device to be detected, and aligning the point cloud model with the second three-dimensional image to obtain a detection image of the device to be detected. Therefore, the present application segments multiple two-dimensional images in the first three-dimensional image of the device to be detected, and then obtains a second three-dimensional image based on the target area image obtained by segmenting multiple two-dimensional images, and then aligns the second three-dimensional image with the point cloud model of the device to be detected to obtain a detection image of the device to be detected. In this way, the detection efficiency and detection accuracy of the device can be improved, and the standardization and scale of device detection can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of a device detection method provided by an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0044] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0045] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0046] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0047] It should be noted that in this article, step codes such as S1 and S2 are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. When implementing the step, those skilled in the art may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.

[0048] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0050] See also Figure 1 , is a flow chart of a device detection method provided in an embodiment of the present application. The device detection method can be executed by an electronic device provided in an embodiment of the present application. The device detection method provided in this embodiment includes:

[0051] Step S1, acquiring a first three-dimensional image of a device to be detected, and segmenting a plurality of two-dimensional images in the first three-dimensional image to obtain target area images in the plurality of two-dimensional images.

[0052] In one embodiment, the first three-dimensional image of the device to be detected can be acquired by using 3D scanning technology. Specifically, the 3D scanning technology can be implemented by devices such as laser scanners, structured light scanners, and 3D cameras.

[0053] In one embodiment, the first three-dimensional image is an image including the device to be detected and background information. The target area image is an image of each area of ​​the device to be detected. It can be understood that the three-dimensional image is actually composed of multiple two-dimensional surfaces, each of which is formed by a frame of two-dimensional images. Therefore, the multiple two-dimensional images can be segmented to filter out the background information except the device to be detected in the multiple two-dimensional images, thereby obtaining the target area image in the multiple two-dimensional images.

[0054] In one embodiment, the Segment Anything Model (SAM) algorithm can be used to segment multiple two-dimensional images in the first three-dimensional image (i.e., coarse segmentation). As a zero-sample segmentation model, the SAM algorithm can be used directly without special training, and can be generalized to a variety of different image types and segmentation tasks. By using the SAM algorithm to segment multiple two-dimensional images, the complexity of data processing can be reduced, and the different regional features of the device to be detected can be identified based on the color of different areas in the two-dimensional image, which not only reduces the demand for computing resources, but also improves the accuracy of segmentation. In other embodiments, segmentation methods such as HRNet (High-Resolution Network), FastSAM, Boundary-aware models, etc. can also be used.

[0055] In one embodiment, the first three-dimensional image has a first three-dimensional coordinate system formed according to pixel points; the multiple two-dimensional images in the first three-dimensional image are segmented in step S1 to obtain target area images in the multiple two-dimensional images, including: obtaining two-dimensional images corresponding to multiple surfaces of the first three-dimensional image according to the two-dimensional coordinates in the first three-dimensional coordinate system; performing plane correction on the two-dimensional images corresponding to the multiple surfaces to obtain the best fitting plane for each surface; and segmenting the best fitting plane to obtain the target area image.

[0056] It can be understood that when a 3D scanner scans a device to be detected, it will generate a corresponding 3D coordinate system based on the 3D image data of the device to be detected and the parameters of the 3D scanner. The 3D coordinate system is a system composed of three mutually perpendicular coordinate axes, namely the X-axis, the Y-axis and the Z-axis. In this system, each axis represents a dimension. The X-axis usually represents the dimension in the horizontal direction, the Y-axis represents another horizontal dimension perpendicular to the X-axis, and the Z-axis represents the vertical dimension perpendicular to the XY plane. Among them, the coordinates of any two dimensions (i.e., two-dimensional coordinates) can determine a plane, i.e., a two-dimensional image. However, during the three-dimensional scanning and imaging process, due to the limitations of the device or the object itself, the acquired two-dimensional image may have certain geometric distortions. Therefore, through plane correction, the image can be transformed into a standard two-dimensional plane coordinate system to eliminate the distortion caused by visual or surface irregularities, so that the features of the device to be detected are clearer and more accurate in the two-dimensional image to improve the accuracy of segmentation. It can be understood that in the best fitting plane obtained by plane correction, the features of the target area will be more obvious, and performing image segmentation based on the best fitting plane can obtain the target area image more efficiently and accurately, thereby improving the efficiency and accuracy of equipment detection.

[0057] In one embodiment, segmenting the best fit plane to obtain a target area image includes: identifying the target area images in multiple best fit planes; and segmenting the target area image according to the two-dimensional coordinates of the target area image to obtain the target area images in multiple best fit planes.

[0058] Specifically, the significant difference between the device to be detected and the background can be identified by a machine learning model, and the key area of ​​the device to be detected can be segmented by a SAM algorithm to obtain a target area image with background information removed. In this way, the computational burden in the subsequent processing steps can be reduced, while retaining sufficient information for use in the subsequent registration process, thereby improving the efficiency and accuracy of device detection. In one embodiment, in step S1, by roughly segmenting multiple two-dimensional images in the first three-dimensional image, an image of the target area in multiple two-dimensional images is obtained, the background information in the first three-dimensional image can be removed, and the image of the information of the device to be detected can be retained, which can provide accurate data for the subsequent registration process, thereby improving the detection efficiency and accuracy.

[0059] Step S2: obtaining a second three-dimensional image of the device to be detected based on the target area image in the multiple two-dimensional images.

[0060] It can be understood that, specifically, the relative position of the target area image obtained by segmentation in the two-dimensional plane can be determined in the three-dimensional image, and then the target area images are combined and output according to the relative position to obtain the second three-dimensional image of the device to be detected after segmentation. In this way, by segmenting and reassembling multiple two-dimensional planes, the interference factors other than the device to be detected in the original first three-dimensional image are removed, and the second three-dimensional image of the device to be detected is obtained, which can prevent the registration process from being interfered by other factors.

[0061] In one embodiment, obtaining a second three-dimensional image of the device to be detected based on the target area images in multiple two-dimensional images in step S2 includes: combining and outputting the target area images in multiple best-fit planes according to the relative positions of the multiple best-fit planes in the first three-dimensional image to obtain a second three-dimensional image of the device to be detected.

[0062] Step S3, modeling the device to be detected based on the first three-dimensional image to obtain a point cloud model of the device to be detected, and aligning the point cloud model with the second three-dimensional image to obtain a detection image of the device to be detected.

[0063] In one embodiment, in step S3, before registering the point cloud data with the second three-dimensional image to obtain the detection result of the device to be detected, the method also includes: identifying local concave parts and local convex parts of the target area image in multiple two-dimensional images, calculating the distance between any first target point in the second three-dimensional image and the convex hull boundary closest to it in the local convex part, so as to judge whether the first target point is located in the local concave part according to the distance; when the distance is greater than a preset distance threshold, determining that the first target point is in the local concave part; when determining that the first target point is in the local concave part, separating the first target point to distinguish different feature areas of the device to be detected, thereby obtaining a segmented second three-dimensional image.

[0064] In this embodiment, when the distance between the first target point and the nearest convex hull boundary in the local convex portion is less than or equal to the preset distance threshold, it is determined that the first target point is located in the local convex portion, and the first target point does not need to be separated.

[0065] It can be understood that the above process describes the fine segmentation of the second three-dimensional image through the concave hull detection algorithm, which specifically includes: identifying the sunken areas (i.e., local concave parts) and convex areas (i.e., local convex parts) in each two-dimensional image in the second three-dimensional image, and separating the sunken areas and convex areas. Specifically, when it is determined that the first target point is located in the local concave part, the first target point is separated from the surrounding area, that is, the first target point is divided out from the area to which it originally belongs. This separation is to distinguish different feature areas during the segmentation of the second three-dimensional image. By identifying the concave parts, different features of the device to be detected (such as grooves, holes, etc.) can be distinguished, so that different areas in the second three-dimensional image can be identified based on different features, thereby improving the detection accuracy.

[0066] Specifically, in this step, the data of the first three-dimensional image of the device to be detected is first converted into a point cloud model of the mold. Before the conversion, the data of the first three-dimensional image can be subjected to noise removal processing, that is, the redundant data points generated in the original data due to scanning errors or external environmental interference are deleted, so that only the data of the three-dimensional image of the device to be detected is retained in the data of the processed first three-dimensional image. Next, the data format of the first three-dimensional image of the device to be detected is converted into a common point cloud format, such as PLY, PCD, ASC and other formats. Then, according to the data of the first three-dimensional image of the device to be detected and the parameters of the scanner, each data point in the first three-dimensional image is mapped to the corresponding three-dimensional space coordinates to form a point cloud model of the device to be detected. Here, the point cloud model of the device to be detected can be completed by three-dimensional modeling or point cloud processing algorithms, such as MeshLab, CloudCompare, Blender and other algorithms to visualize the point cloud model. In addition, in order to improve the quality of the point cloud model so that the subsequent registration process has higher accuracy, optionally, a smoothing algorithm can be used to improve the appearance of the point cloud model, remove unnecessary details or strengthen necessary details, so that it can more accurately express the three-dimensional form of the device to be detected.

[0067] In this embodiment, registering the point cloud model with the second three-dimensional image means aligning the point cloud model of the device to be detected and the second three-dimensional image into the same coordinate system so that they correspond in space. The purpose is to identify and minimize the difference between the point cloud model and the second three-dimensional image to ensure the accuracy of the detection.

[0068] In one embodiment, the step S3 of registering the point cloud data with the second three-dimensional image to obtain the detection result of the device to be detected includes: obtaining a second target point corresponding to the first target point in the second three-dimensional image from the point cloud model, and obtaining a rotation matrix that minimizes the distance between the first target point and the second target point; iteratively calculating the rotation matrix so that the rotation matrix reaches a preset convergence standard; when the rotation matrix reaches the preset convergence standard, the corresponding second three-dimensional image is determined as the detection image of the device to be detected.

[0069] Specifically, the registration process between the point cloud model and the second three-dimensional image may include: extracting feature points for registration from the point cloud model and the second three-dimensional image; determining the correspondence between the feature points in the point cloud model and the second three-dimensional image; calculating transformation parameters that make the point cloud model and the corresponding feature points in the second three-dimensional image as close as possible, the transformation parameters including a rotation matrix and a translation vector; and iteratively optimizing the transformation parameters to reduce the registration error.

[0070] Optionally, the registration process can be performed by an ICP (Iterative Closest Point) algorithm. The registration process is actually to find the best alignment between the two data sets (i.e., the point cloud model and the second three-dimensional image), by calculating the rotation matrix and translation vector that minimize the distance between the first target point and the second target point, thereby transforming the point cloud model to a state that is as aligned as possible with the second three-dimensional image.

[0071] Through iterative calculation, the rotation matrix and translation vector are continuously updated to minimize the distance between the first target point and the second target point. When the iterative process reaches the preset convergence criterion, the algorithm stops iterating. When the convergence criterion is reached, the points in the point cloud model are transformed to positions aligned with the corresponding points in the second three-dimensional image using the final calculated rotation matrix and translation vector. At this time, the second three-dimensional image aligned with the transformed point cloud model is considered to be the detection image of the device to be detected. In this state, the points in the second three-dimensional image and the points in the point cloud have achieved the best match in space, so the second three-dimensional image reflects the actual state of the device to be detected. By accurately aligning the point cloud model and the second three-dimensional image, accurate alignment ensures the consistency of the point cloud model and the second three-dimensional image, which can improve the quality and efficiency of equipment detection.

[0072] In one embodiment, the iterative calculation of the transformation matrix so that the transformation matrix reaches a preset convergence standard includes:

[0073] Iteratively updating the rotation matrix and the translation vector to reduce the error between the first target point and the second target point;

[0074] The error is calculated by the following formula:

[0075]

[0076] Where J is the error between the first target point and the second target point, p i represents the first target point, q i represents the second target point, i represents the point cloud sequence number in the point cloud model and the second three-dimensional image, n represents the total number of point clouds in the point cloud model, R is the rotation matrix, and t is the translation vector, which represents the translation required to align the second target point with the third target point.

[0077] It can be understood that by reducing the error in the registration process, the accuracy of the registration can be improved to ensure accurate alignment between the point cloud model and the second three-dimensional image, and the smaller the error in the registration process, the more reliable the registration result, which means that the detection result is more accurate.

[0078] In one embodiment, in step S1, obtaining a first three-dimensional image of the device to be detected includes:

[0079] A three-dimensional scanner is used to scan the device to be detected at multiple angles to obtain corresponding scanning results; the scanning results are fused to obtain the three-dimensional coordinates of each point on the surface of the device to be detected; and a first three-dimensional image is obtained based on the three-dimensional coordinates.

[0080] It is understandable that when the device to be detected is a large mold or a mold with a relatively complex shape, multi-angle scanning is required to ensure that the characteristic information of each surface of the device to be detected can be detected, so as to obtain high-quality point cloud data to ensure the accuracy of the detection. In addition, when the shape of the device to be detected is relatively simple, multi-angle scanning may not be used, and one or two angles may be determined for scanning. The scanning angle and number of scans are determined according to the shape, size and other characteristics of the device to be detected, which can save processing resources and improve detection efficiency.

[0081] The device detection method provided by the present invention uses a 3D scanning device to obtain a first three-dimensional image of the device to be detected, uses an algorithm to perform a first segmentation on each two-dimensional image of the three-dimensional image, and then reassembles the target area image obtained by the segmentation to obtain a second three-dimensional image of the device to be detected, models the device to be detected through the first three-dimensional image to obtain a point cloud model, and aligns the second three-dimensional image with the point cloud model to obtain a detection result. It can improve the detection efficiency and detection accuracy of the device and realize the standardization and scale of equipment detection.

[0082] Based on the same inventive concept as the above embodiments, an embodiment of the present invention provides an electronic device, such as Figure 2 As shown, the electronic device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 2The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Figure 2 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the above device detection method is implemented.

[0083] The electronic device may also include: at least one network interface 312. The various components in the electronic device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 313 is not used in the following examples. Figure 2 Various buses are labeled as bus system 313.

[0084] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct memory bus random access memory (DRRAM). The memory 311 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memory.

[0085] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device. Examples of these data include: any computer program used to operate on the electronic device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0086] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer-readable storage medium, in which a computer program is stored, and the computer-readable storage medium may be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM), etc.; or it may be various devices including one or any combination of the above-mentioned memories, such as a server, etc. When the instructions stored therein are executed by a processor, the device detection method described in the above-mentioned embodiment is implemented.

[0087] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0089] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A device detection method, characterized in that: include: Acquire a first three-dimensional image of the device to be detected, and segment a plurality of two-dimensional images in the first three-dimensional image to obtain target area images in the plurality of two-dimensional images; Based on the target area image in the multiple two-dimensional images, obtaining a second three-dimensional image of the device to be detected; The device to be detected is modeled based on the first three-dimensional image to obtain a point cloud model of the device to be detected, and the point cloud model is registered with the second three-dimensional image to obtain a detection image of the device to be detected.

2. The method according to claim 1, characterized in that The first three-dimensional image has a first three-dimensional coordinate system formed according to pixel points; The step of segmenting the plurality of two-dimensional images in the first three-dimensional image to obtain target area images in the plurality of two-dimensional images includes: Obtaining two-dimensional images corresponding to a plurality of surfaces of the first three-dimensional image according to the two-dimensional coordinates in the first three-dimensional coordinate system; Performing plane correction on the two-dimensional images corresponding to the multiple surfaces to obtain the best fitting plane for each surface; The best fitting plane is segmented to obtain the target area image.

3. The method according to claim 2, characterized in that The step of segmenting the best fitting plane to obtain the target area image includes: identifying a target region in the plurality of best-fit planes; The target area is segmented according to the two-dimensional coordinates of the target area to obtain target area images in the multiple best fitting planes.

4. The method according to claim 3, characterized in that The step of obtaining a second three-dimensional image of the device to be detected based on the target area image in the multiple two-dimensional images includes: The target area images in the multiple best-fit planes are combined and output according to the relative positions of the multiple best-fit planes in the first three-dimensional image to obtain a second three-dimensional image of the device to be detected.

5. The method according to claim 1, characterized in that Before registering the point cloud data with the second three-dimensional image to obtain the detection result of the device to be detected, the method further includes: Identify local concave parts and local convex parts of the target area image in the multiple two-dimensional images, and calculate the distance between any first target point in the second three-dimensional image and the convex hull boundary closest to it, so as to determine whether the first target point is located in the local concave part according to the distance; When the distance is greater than a preset distance threshold, determining that the first target point is in a local concave portion; When it is determined that the first target point is located in a local concave portion, the first target point is separated to distinguish different characteristic regions of the device to be detected, thereby obtaining a segmented second three-dimensional image.

6. The method according to any one of claims 1 to 5, characterized in that: The registering the point cloud data with the second three-dimensional image to obtain the detection result of the device to be detected includes: Acquire a second target point corresponding to the first target point in the second three-dimensional image from the point cloud model, and acquire a rotation matrix that minimizes the distance between the first target point and the second target point; Iteratively calculating the rotation matrix so that the rotation matrix reaches a preset convergence standard; When the rotation matrix reaches a preset convergence standard, the corresponding second three-dimensional image is determined as the detection image of the device to be detected.

7. The method according to claim 6, characterized in that The iterative calculation of the transformation matrix so that the transformation matrix reaches a preset convergence standard includes: Iteratively updating the rotation matrix and the translation vector to reduce the error between the first target point and the second target point; The error is calculated by the following formula: Where J is the error between the first target point and the second target point, p i represents the first target point, q i represents the second target point, i represents the point cloud sequence number in the point cloud model and the second three-dimensional image, n represents the total number of point clouds in the point cloud model, R is the rotation matrix, and t is the translation vector, which represents the translation required to align the second target point with the third target point.

8. The method according to claim 1, characterized in that: The step of acquiring a first three-dimensional image of the device to be detected includes: Use a three-dimensional scanner to scan the device to be detected at multiple angles to obtain corresponding scanning results; The scanning results are merged to obtain the three-dimensional coordinates of each point on the surface of the device to be detected; The three-dimensional image is obtained based on the three-dimensional coordinates.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the device detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor, the device detection method according to any one of claims 1 to 8 is implemented.

Images