Symmetry detection method and device and computer equipment

Abstract

The invention relates to a symmetry detection method and device and computer equipment. The method comprises the following steps: acquiring a grid model of a detection target, wherein the grid model comprises first three-dimensional point cloud information matched with a to-be-detected part in the detection target; generating mirror image three-dimensional point cloud information according to the first three-dimensional point cloud information; determining second three-dimensional point cloud information matched with a symmetrical part of the to-be-detected part according to the mirror image three-dimensional point cloud information; and determining a symmetry detection result of the to-be-detected part according to a matching result of the mirror image three-dimensional point cloud information and the second three-dimensional point cloud information. On one hand, the three-dimensional point cloud information of the detection target is acquired and processed, so that the symmetry of the to-be-detected part of the detection target can be determined more accurately; and on the other hand, the symmetry detection efficiency is effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular to a symmetry detection method, apparatus, and computer equipment. Background Art

[0002] The detection and analysis of symmetry in biological parts is an important research area in biomechanical analysis. For example, in human morphological assessment, symmetry is a key indicator for evaluating individual health and balance. Symmetry not only affects appearance but is also closely related to physical function and athletic ability. Most biological parts are generally symmetrical, such as the shoulders, knees, and hips in humans and animals. However, in some cases of disease, injury, or developmental abnormalities, body symmetry can be affected. Therefore, symmetry testing has broad applications in physical rehabilitation, biomechanical assessment, and medicine.

[0003] However, traditional symmetry assessment techniques rely on subjective observations of the target based on human experience to assess the symmetry of specific parts of the target. This can be inaccurate, and manual evaluation is inefficient. Summary of the Invention

[0004] Based on this, it is necessary to provide a symmetry detection method, device, and computer equipment that can improve detection accuracy and detection efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a symmetry detection method. The method comprises:

[0006] Acquire a grid model of a detection target, wherein the grid model includes first three-dimensional point cloud information matching a part to be detected in the detection target;

[0007] generating mirrored three-dimensional point cloud information based on the first three-dimensional point cloud information;

[0008] Determining, based on the mirrored three-dimensional point cloud information, second three-dimensional point cloud information that matches a symmetrical portion of the portion to be detected;

[0009] A symmetry detection result of the part to be detected is determined according to a matching result between the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information.

[0010] In one embodiment, generating mirrored three-dimensional point cloud information based on the first three-dimensional point cloud information includes:

[0011] Acquiring posture information of the detection target;

[0012] Determine a local point cloud coordinate system according to the first three-dimensional point cloud information and the posture information;

[0013] According to the x-axis direction of the local point cloud coordinate system, the mirror coordinates of the first three-dimensional point cloud information coordinates are determined, and the mirror three-dimensional point cloud information is determined according to the mirror coordinates.

[0014] In one embodiment, determining a local point cloud coordinate system according to the first three-dimensional point cloud information and the posture information includes:

[0015] Determining the origin of the local point cloud coordinate system according to the centroid point cloud information in the first three-dimensional point cloud information;

[0016] determining a z-axis direction of the local point cloud coordinate system according to the direction of the first three-dimensional point cloud information;

[0017] Determine, based on the posture information, a projection component perpendicular to the ground projected onto the three-dimensional point cloud information plane, and determine the projection component perpendicular to the z-axis as the x-axis direction of the local point cloud coordinate system;

[0018] Determine the y-axis direction based on the perpendicular direction of the plane formed by the x-axis and the z-axis;

[0019] The local point cloud coordinate system is obtained according to the origin, x-axis direction, y-axis direction, and z-axis direction.

[0020] In one embodiment, obtaining the local point cloud coordinate system according to the origin, the x-axis direction, the y-axis direction, and the z-axis direction includes:

[0021] Determine a main direction of the first three-dimensional point cloud information as the positive z-axis direction of the local point cloud coordinate system;

[0022] Determine the positive direction of the y-axis of the local point cloud coordinate system, where the angle between the positive direction of the y-axis and the vertical direction of the ground is less than 90 degrees;

[0023] The positive direction of the x-axis is determined according to the positive direction of the z-axis and the positive direction of the y-axis, and the local point cloud coordinate system is determined according to the positive direction of the x-axis, the positive direction of the y-axis, and the positive direction of the z-axis.

[0024] In one embodiment, determining the symmetry detection result of the to-be-detected portion according to the matching result of the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information includes:

[0025] Determining, based on the matching result and a preset accuracy value, a coordinate difference value between a mirrored 3D point in the mirrored 3D point cloud information and a corresponding 3D point in the second 3D point cloud information in a target coordinate axis direction;

[0026] A symmetry detection result of the part to be detected is determined in the grid model according to the coordinate difference value and the visual distinguishing mark.

[0027] In one embodiment, the visual distinguishing mark includes a preset color, and determining the symmetry detection result of the to-be-detected portion in the grid model according to the coordinate difference value and the visual distinguishing mark includes:

[0028] If the coordinate difference value is greater than 0, the corresponding mirrored three-dimensional point is set to the first color, and the size of the difference value is positively correlated with the depth of the first color;

[0029] If the coordinate difference value is less than 0, the corresponding mirrored 3D point is set to a second color, and the size of the difference value is negatively correlated with the depth of the second color;

[0030] The mirrored three-dimensional points after being set with the first color or the second color are superimposed on the grid model, and the symmetry detection result is determined based on the superimposed grid model.

[0031] In a second aspect, the present application further provides a symmetry detection method, the method comprising:

[0032] In response to a selection instruction of an input device, determining a portion of a detection target to be detected;

[0033] Determining first three-dimensional point cloud information according to the part to be detected;

[0034] Obtaining second three-dimensional point cloud information based on the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information;

[0035] The symmetry detection result is displayed through a display device; the symmetry detection result is determined according to the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information and the second three-dimensional point cloud information.

[0036] In one embodiment, after displaying the symmetry detection result on a display device, the method further includes:

[0037] In response to a movement instruction of the input device, the mirrored three-dimensional point is moved to obtain updated mirrored three-dimensional point cloud information;

[0038] Determining an updated coordinate difference value in the direction of the target coordinate axis between the updated mirrored three-dimensional point cloud information and the corresponding three-dimensional point in the second three-dimensional point cloud information;

[0039] The updated symmetry detection result of the part to be detected is displayed on the display device according to the updated coordinate difference value and the updated visual distinguishing mark.

[0040] In a third aspect, the present application further provides a symmetry detection device. The device comprises:

[0041] A grid model acquisition module, configured to acquire a grid model of a detection target, wherein the grid model includes first three-dimensional point cloud information matching a part to be detected in the detection target;

[0042] a mirrored three-dimensional point cloud information generating module, configured to generate mirrored three-dimensional point cloud information based on the first three-dimensional point cloud information;

[0043] A second three-dimensional point cloud information generating module is used to determine second three-dimensional point cloud information that matches a symmetrical portion of the portion to be detected based on the mirrored three-dimensional point cloud information;

[0044] The symmetry detection module is used to determine the symmetry detection result of the part to be detected based on the matching result of the mirror three-dimensional point cloud information and the second three-dimensional point cloud information.

[0045] In a fourth aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the computer device implements the steps of any one of the symmetry detection methods in the first aspect.

[0046] The above-mentioned symmetry detection method, device, and computer equipment realize automatic detection of symmetry by obtaining a grid model of the detection target and determining the symmetry detection result of the part to be detected based on the mirror matching method of the grid model point cloud information. On the one hand, by utilizing the three-dimensional point cloud information of the detection target for processing, the symmetry of the part to be detected of the detection target can be determined more accurately; on the other hand, the efficiency of symmetry detection is also effectively improved.

[0047] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0049] Figure 1 A diagram illustrating an application environment of a symmetry detection method according to an embodiment;

[0050] Figure 2 1 is a flow chart of a symmetry detection method according to an embodiment;

[0051] Figure 3is a schematic diagram of filling holes in a grid model when the spacing is less than a first threshold in a specific embodiment;

[0052] Figure 4 is a schematic diagram of filling holes in a grid model when the spacing is greater than or equal to a first threshold in a specific embodiment;

[0053] Figure 5 is a structural block diagram of a symmetry detection device in one embodiment;

[0054] Figure 6 is a structural block diagram of a symmetry detection device in another embodiment;

[0055] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] The terms "module", "unit", etc. used below refer to a combination of software and / or hardware that can implement a predetermined function. Although the devices described in the following embodiments are preferably implemented in hardware, implementation using software or a combination of software and hardware is also possible and contemplated.

[0058] The symmetry detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 detects the grid model of the target and sends it to the server 104, and the grid model includes a first three-dimensional point cloud information that matches the part to be detected in the detection target. The server 104 generates mirrored three-dimensional point cloud information based on the first three-dimensional point cloud information; determines the second three-dimensional point cloud information that matches the symmetrical part of the part to be detected based on the mirrored three-dimensional point cloud information; and determines the symmetry detection result of the part to be detected based on the matching result of the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information and sends it to the terminal 102. The execution steps of the above-mentioned symmetry detection method can also be performed separately by the server 104 or the terminal 102, and this application does not limit this. Among them, the server 104 can be implemented as an independent server or a server cluster composed of multiple servers.

[0059] In one embodiment, Figure 2As shown, a symmetry detection method is provided, which is applied to Figure 1 The application scenario in the example is used to illustrate the following steps:

[0060] S201: Acquire a grid model of a detection target, where the grid model includes first three-dimensional point cloud information matching a part to be detected in the detection target.

[0061] In an embodiment of the present application, the detection target may include a biological target to be detected, such as a human body, an animal body, a plant body, etc. Acquiring a grid model of the detection target may include acquiring multiple images of the detection target, performing three-dimensional reconstruction on the multiple images to obtain overall three-dimensional point cloud information of the detection target, and generating a grid model based on the overall three-dimensional point cloud information. The multiple images of the detection target may include RGB images of the detection target at different angles, and the multiple images of the detection target may be acquired by an RGB camera device.

[0062] The following describes a method for determining the first three-dimensional point cloud information through an embodiment of the present application.

[0063] In an embodiment of the present application, after obtaining a grid model, a target portion to be inspected can be selected within the grid model. For example, an input device such as a mouse or a drawing board can be used to circle a portion of the grid model where symmetry testing is required, such as the neck and shoulder area or the right knee. After determining the target portion and its symmetrical portion within the grid model, first three-dimensional point cloud information matching the target portion can be determined based on the mapping relationship between the grid model and the three-dimensional point cloud information.

[0064] In some embodiments, a first local mesh model that matches the part to be inspected can be determined within the mesh model, and then the first three-dimensional point cloud information corresponding to the first local mesh model can be determined. In this embodiment of the present application, since the mesh model is constructed based on the three-dimensional point cloud information, the first three-dimensional point cloud information that matches the part to be inspected within the inspection target can be determined based on the mapping relationship between the mesh model and the three-dimensional point cloud information.

[0065] Furthermore, based on a real-time target detection algorithm, low-texture image matching can be performed on multiple images through dense feature matching to achieve high-precision 3D reconstruction of the multiple images, obtaining overall 3D point cloud information of the detected target, and then determining a mesh model. The overall 3D point cloud information includes the 3D point cloud information of the entire detected target. It is understood that the mesh model determined based on the overall 3D point cloud information also corresponds to the mesh model of the entire detected target.

[0066] S203: Generate mirrored three-dimensional point cloud information according to the first three-dimensional point cloud information.

[0067] S205: Determine second three-dimensional point cloud information that matches a symmetrical portion of the portion to be detected based on the mirrored three-dimensional point cloud information.

[0068] In the embodiment of the present application, based on the first three-dimensional point cloud information corresponding to the part to be inspected, a mirror image three-dimensional point cloud information can be determined through a mirror transformation. Specifically, the x-axis coordinate of each three-dimensional point in the first three-dimensional point cloud information can be inverted to determine the mirror image three-dimensional point cloud information. Furthermore, a point cloud information matching result can be determined between the mirror image three-dimensional point cloud information and the second three-dimensional point cloud information corresponding to the symmetrical part.

[0069] It can be understood that the mirrored three-dimensional point cloud information represents the position information of the symmetrical parts of the part to be detected in theory, and the second three-dimensional point cloud information represents the position information of the symmetrical parts of the part to be detected in the actual detection target. The point cloud information matching result can represent the position difference between the actual symmetrical parts and the theoretical symmetrical parts.

[0070] S207: Determine a symmetry detection result of the part to be detected according to a matching result between the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information.

[0071] It is understood that after determining the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information, the point cloud information can be matched to determine the matching result. After determining the point cloud information matching result, the symmetry detection result of the part to be detected can be determined and displayed on the grid model. Specifically, the symmetry detection result of the detection target can be intuitively displayed on the grid model by using different visual distinguishing marks; it is also possible to choose to display the symmetry detection result on any one of the coordinate axes of the x-axis, y-axis, and z-axis through the grid model, or it is possible to choose to display the symmetry detection result on the x-axis and y-axis, the x-axis and z-axis, the y-axis and z-axis, or the x-axis, y-axis, and z-axis through the grid model. For example, the coordinate value difference between the mirrored three-dimensional point cloud information and the matched second three-dimensional point cloud information is determined as the matching result, such as the x-axis coordinate value difference, the y-axis coordinate value difference, or the z-axis coordinate value difference. If the coordinate value difference is less than a preset threshold, the detection part represented by the point cloud information is considered symmetrical. If the coordinate value difference is not less than the preset threshold, the detection part represented by the point cloud information is considered asymmetrical. In other embodiments, the geometric distance between the mirrored three-dimensional point cloud information and the matched second three-dimensional point cloud information can also be used as a matching result. If the geometric distance is less than a preset distance threshold, the detection part represented by the point cloud information is considered to be symmetrical; if the geometric distance is not less than the distance threshold, the detection part represented by the point cloud information is considered to be asymmetrical.

[0072] Furthermore, the visual distinguishing mark may include a preset color or preset brightness information. Specifically, the asymmetric point cloud information may be marked with a preset color, and ultimately all asymmetric point cloud information may be displayed with the preset color as the symmetry detection result. In other embodiments, the symmetry detection result may also be displayed by brightness. For example, if the coordinate value difference is not less than a preset threshold, the detection part represented by the point cloud information is considered asymmetric, and the asymmetric point cloud information may be displayed with high brightness, and ultimately all asymmetric point cloud information may be highlighted as the symmetry detection result.

[0073] The symmetry detection method provided in the embodiment of the present application realizes automatic detection of symmetry by obtaining a grid model of the detection target and determining the symmetry detection result of the part to be detected based on the mirror matching method of the grid model point cloud information. On the one hand, by utilizing the three-dimensional point cloud information of the detection target for processing, the symmetry of the part to be detected of the detection target can be determined more accurately; on the other hand, the efficiency of symmetry detection is also effectively improved.

[0074] This application combines an IMU with an RGB camera to determine three-dimensional point cloud information by acquiring a dense point cloud, effectively improving the accuracy and robustness of the three-dimensional point cloud information. The real-time performance of symmetry detection can be improved by acquiring SFM (Structure from Motion) feature points. In addition, this application provides a quantitative detection method that can intuitively observe the symmetry detection results through color information or brightness information, and allows users to select the part to be detected as needed, as well as adjust the detection results, thereby improving the accuracy and efficiency of symmetry detection.

[0075] Furthermore, the present application can perform symmetry detection through common non-specific equipment, and does not require professional equipment to obtain medical images or scanners, and has universality and wider application. On the other hand, the present application can perform symmetry detection on any part of the detection target, for example, it can detect the hairline, eyebrows, cheekbones, lips, jaw line, clavicle, breasts, buttocks and other parts, with high flexibility, and does not require key point identification and segmentation of the parts to be detected. The present application can also interactively adjust the symmetry detection results, and the user can adjust the symmetrical grid model and perform real-time symmetry detection. The present application is also universal and real-time, and can provide real-time guidance for doctors during surgery, and can also be used by users for real-time self-detection.

[0076] In some embodiments, generating mirrored three-dimensional point cloud information based on the first three-dimensional point cloud information includes:

[0077] S301: Acquire posture information of the detection target.

[0078] S303: Determine a local point cloud coordinate system according to the first point cloud information and the posture information.

[0079] S305: Determine the mirror coordinates of the first three-dimensional point cloud information coordinates according to the x-axis direction of the local point cloud coordinate system, and determine the mirror three-dimensional point cloud information according to the mirror coordinates.

[0080] In an embodiment of the present application, posture information of the detection target is obtained, and the posture information may include IMU (Inertial Measurement Unit) information. A local point cloud coordinate system can be constructed based on the first three-dimensional point cloud information, and the direction of the local coordinate system axis can be further determined based on the posture information. Based on the x-axis direction of the local point cloud coordinate system, the mirror coordinates of the first three-dimensional point cloud information coordinates can be determined, and then the mirrored three-dimensional point cloud information can be determined.

[0081] In an embodiment of the present application, a local point cloud coordinate system is determined based on the first point cloud information and the posture information, and then the mirror coordinates of the first three-dimensional point cloud information coordinates are determined, thereby ensuring that the directionality of the mirror operation is consistent with the actual physical space, avoiding the mirror deviation that may be caused by relying solely on the geometric features of the point cloud information, and thus making the mirrored three-dimensional point cloud information more accurately reflect the posture information of the detection target.

[0082] The following describes a specific method for determining a local point cloud coordinate system through an embodiment of the present application. In some embodiments, determining a local point cloud coordinate system based on the first three-dimensional point cloud information and the posture information includes:

[0083] S401: Determine the origin of the local point cloud coordinate system according to the centroid point cloud information in the first three-dimensional point cloud information.

[0084] S403: Determine the z-axis direction of the local point cloud coordinate system according to the direction of the first three-dimensional point cloud information.

[0085] S405: Determine a projection component perpendicular to the ground and projected onto the three-dimensional point cloud information plane according to the posture information, and determine a projection component perpendicular to the z-axis as the x-axis direction of the local point cloud coordinate system.

[0086] S407: Determine the y-axis direction according to the perpendicular direction of the plane formed by the x-axis and the z-axis.

[0087] S409: Obtain the local point cloud coordinate system according to the origin, x-axis direction, y-axis direction, and z-axis direction.

[0088] In this embodiment of the present application, the average of the three-dimensional coordinates of each three-dimensional point in the first three-dimensional point cloud information is determined as the coordinates of the centroid three-dimensional point and used as the origin of the local point cloud coordinate system. The z-axis direction of the local point cloud coordinate system can be determined based on the direction of the first three-dimensional point cloud information.

[0089] The direction perpendicular to the ground can be determined based on the posture information. The direction perpendicular to the ground is projected onto the three-dimensional point cloud information plane to determine a projection component, and the projection component perpendicular to the z-axis is determined as the x-axis direction of the local point cloud coordinate system.

[0090] The y-axis direction of the local point cloud coordinate system is determined according to the perpendicular direction of the xz plane formed by the x-axis and the z-axis. The local point cloud coordinate system can then be obtained according to the origin, the x-axis direction, the y-axis direction, and the z-axis direction.

[0091] This embodiment of the application uses the center of mass as the origin to ensure spatial consistency, combines the point cloud's principal direction to determine the z-axis, and uses posture projection orthogonality to derive the x-axis direction, ultimately obtaining the local point cloud coordinate system. This significantly improves the adaptability of the local point cloud coordinate system to the true posture of the detected target. It also provides a high-precision spatial reference for subsequent determination of mirror coordinates through mirror transformation.

[0092] In some embodiments, the positive directions of the x-axis, y-axis, and z-axis may also be determined. In some embodiments, obtaining the local point cloud coordinate system based on the origin, the x-axis direction, the y-axis direction, and the z-axis direction includes:

[0093] S501: Determine the main direction of the first three-dimensional point cloud information as the positive direction of the z-axis of the local point cloud coordinate system.

[0094] S503: Determine the positive direction of the y-axis of the local point cloud coordinate system, where the angle between the positive direction of the y-axis and the vertical direction of the ground is less than 90 degrees.

[0095] S505: Determine the positive direction of the x-axis according to the positive direction of the z-axis and the positive direction of the y-axis, and determine the local point cloud coordinate system according to the positive direction of the x-axis, the positive direction of the y-axis, and the positive direction of the z-axis.

[0096] In an embodiment of the present application, the principal direction of the first three-dimensional point cloud information can be determined using a principal component analysis (PCA) algorithm. Specifically, the covariance matrix of the first three-dimensional point cloud information is calculated, and the covariance matrix is ​​subjected to eigenvalue decomposition to obtain eigenvalues ​​and eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue is the principal direction of the first three-dimensional point cloud information, that is, the maximum direction of the first three-dimensional point cloud information distribution. The principal direction is used as the positive z-axis direction of the local point cloud coordinate system.

[0097] Furthermore, the positive direction of the y-axis can be determined by setting the angle between the positive direction of the y-axis and the vertical direction to no more than 90 degrees. Using the right-hand coordinate system rule and the determined positive directions of the y-axis and z-axis, the positive direction of the x-axis can be determined. The local point cloud coordinate system can then be determined based on the positive directions of the x-axis, y-axis, and z-axis.

[0098] In the embodiment of the present application, by confirming the positive directions of the x-axis, y-axis, and z-axis, on the one hand, the method of mirroring the data can be determined, and on the other hand, it can also facilitate the subsequent assignment of visual distinguishing marks. The positive direction of the x-axis is determined by the positive direction of the z-axis and the direction of gravity of the IMU, which can ensure that the positive direction of the x-axis is perpendicular to the ground; the positive direction of the y-axis and the direction of gravity of the IMU do not exceed 90 degrees, in order to ensure that the positive direction of the x-axis points to the right side of the data itself (i.e., the left side of the viewing angle). The definition of the positive direction of the unified coordinate axis can ensure that the subsequent visual distinguishing marks are correctly displayed.

[0099] In some embodiments, the symmetry detection result can be determined in real time in an intuitive manner. The determination of the symmetry detection result of the part to be detected based on the matching result of the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information includes:

[0100] S601: Determine, based on the matching result and a preset accuracy value, a coordinate difference value between a mirrored 3D point in the mirrored 3D point cloud information and a corresponding 3D point in the second 3D point cloud information in a target coordinate axis direction.

[0101] S603: Determine the symmetry detection result of the part to be detected in real time in the grid model according to the coordinate difference value and the visual distinguishing mark.

[0102] In an embodiment of the present application, based on the point cloud information matching result and the preset accuracy value, the coordinate difference value between the mirrored three-dimensional point in the mirrored three-dimensional point cloud information and the corresponding three-dimensional point in the second three-dimensional point cloud information in the direction of the target coordinate axis can be determined. The target coordinate axis can be the x-axis, y-axis or z-axis of the local point cloud coordinate axis. In some embodiments, the visual distinguishing mark may include a preset color. Specifically, the absolute value of the coordinate difference value is divided by the preset accuracy value to determine the target value, and then the target value is multiplied by the pixel value of the preset color. The symmetry detection result of the part to be detected can be mapped to the corresponding color, and the greater the symmetry difference in the symmetry detection result, the darker the corresponding preset color.

[0103] In other embodiments, the visually distinguishable mark may further include preset brightness information. Specifically, the absolute value of the coordinate difference value may be divided by a preset accuracy value to determine a target value, which is then multiplied by a preset brightness coefficient to map the symmetry test result of the part to be tested to corresponding brightness information. The greater the symmetry difference in the symmetry test result, the higher the corresponding brightness.

[0104] In the embodiment of the present application, the coordinate difference value along the target coordinate axis is determined based on the matching result and a preset precision value, and the symmetry detection result is displayed with a visual visual marker based on the coordinate difference value and the visual distinguishing mark, thereby achieving visual real-time feedback of the symmetry detection result. This application not only improves the interpretability of the symmetry detection results and can efficiently locate the asymmetric parts of the detection target, but also adapts to the sensitivity requirements of different symmetry detection scenarios through intuitive expressions such as parameterized precision values ​​and visual distinguishing marks.

[0105] The following examples of the present application illustrate an implementation method for displaying symmetry detection results in preset colors. In some embodiments, the visual distinguishing mark includes a preset color, and the real-time determination of the symmetry detection result of the to-be-detected portion in the grid model based on the coordinate difference value and the visual distinguishing mark includes:

[0106] S701: If the coordinate difference value is greater than 0, set the corresponding mirrored three-dimensional point to a first color, and the size of the difference value is positively correlated with the depth of the first color.

[0107] S703: If the coordinate difference value is less than 0, set the corresponding mirrored three-dimensional point to a second color, and the size of the difference value is negatively correlated with the depth of the second color.

[0108] S705: Superimposing the mirrored three-dimensional points after setting the first color or the second color onto the grid model, and determining the symmetry detection result according to the superimposed grid model.

[0109] In an embodiment of the present application, if the coordinate difference value is greater than 0, the corresponding mirrored three-dimensional point is set to the first color, and the size of the difference value is positively correlated with the depth of the first color; if the coordinate difference value is less than 0, the corresponding mirrored three-dimensional point is set to the second color, and the size of the difference value is negatively correlated with the depth of the second color.

[0110] The mirrored three-dimensional points, after being set to the first or second color, are superimposed on the grid model, and the symmetry detection results are determined and displayed intuitively based on the superimposed grid model. This application achieves a visual display of the symmetry detection results by mapping the coordinate difference values ​​to different color settings, thereby more intuitively showing the asymmetric parts of the detection target, effectively improving the real-time and interactivity of symmetry detection.

[0111] The following describes a method for obtaining a grid model using an embodiment of the present application. In some embodiments, obtaining a grid model of a detection target includes:

[0112] S801: Determine a target detection area of ​​the image of each detection target, and generate a grayscale image of the image of each detection target according to each target detection area.

[0113] S803: Taking every two grayscale images as a fixed image and a moving image, matching the dense feature points of the fixed image and the moving image according to a preset feature matching algorithm, and determining the dense feature matching result.

[0114] S805: Perform three-dimensional reconstruction based on the dense feature matching results and the epipolar constraint relationship to determine the overall three-dimensional point cloud information of the detection target, and generate the grid model based on the overall three-dimensional point cloud information.

[0115] In an embodiment of the present application, the overall three-dimensional point cloud information includes the three-dimensional point cloud information of the detection target as a whole. It can be understood that the grid model determined based on the overall three-dimensional point cloud information is also the corresponding grid model of the detection target as a whole. Targets in multiple images can be detected by a preset target detection algorithm. Specifically, the target detection model YOLOv5 can be used to determine the target organism in multiple images, and the area unrelated to the target organism can be cropped according to the detection frame of the target organism to reduce the interference of irrelevant features and improve the accuracy of target feature matching. Dense feature matching is performed on the cropped multiple images, and three-dimensional reconstruction is performed based on the dense feature matching results and the polar constraint relationship to determine the three-dimensional point cloud information.

[0116] In the embodiment of the present application, a target detection area in each image of a detection target is determined by a preset target detection algorithm, and a grayscale image of each detection target image is generated for the target detection area in each image. It is understood that the number of detection target images matches the number of grayscale images.

[0117] The preset feature matching algorithm may include LoFTR (Local Feature Transformer, a feature matching algorithm), which takes every two grayscales as the fixed image (fixed image) and the moving image (moved image) of LoFTR, and uses the weights trained in indoor scenes as the network weights to match the dense feature points of the fixed image and the moving image to determine the dense feature matching results. In the dense feature matching process, the fixed image is usually a static reference frame image, and the moving image is usually the image to be matched. The corresponding feature point pairs in the two images can be determined by comparison. Furthermore, the LoFTR algorithm integrates the global information of the two grayscale images according to the self-attention mechanism, captures the long-distance dependencies in the image, and outputs the dense matching results of the two grayscale images. The LoFTR algorithm not only matches the key points in the image, but also finds corresponding matching points for most pixels in the image, and can find effective matching feature points in weak texture areas.

[0118] In an embodiment of the present application, three-dimensional reconstruction can perform feature matching on the RGB images collected by the device through the detect-LoFTR algorithm, and filter out the feature points of the target area to achieve image feature matching. The image is then classified according to the camera pose information, and feature trajectories are constructed within and between categories. The sparse reconstruction of the scene is then completed by triangulation and BA optimization using the feature trajectory and camera pose information. Finally, based on the sparse reconstruction results, the NCC indicator (Normalized Cross Correlation) is used for depth estimation and point cloud propagation to obtain a dense point cloud. Among them, BA optimization (Bundle Adjustment) is a core optimization technology in computer vision, which is mainly used to jointly optimize the poses of multi-frame cameras and the coordinate parameters of three-dimensional space points by minimizing the reprojection error.

[0119] In the embodiment of the present application, a feature screening module is added on the basis of the LoFTR algorithm, which can focus on the feature matching of the target area and improve the efficiency and accuracy of the matching. The camera pose information is used as a clustering feature to construct a feature trajectory based on the candidate image, which improves the stability and accuracy of the feature matching. On the other hand, the present application uses the device camera pose and internal reference information for scaling, avoiding the need for additional spatial information, simplifying the reconstruction process, and using NCC as a consistency measurement indicator to optimize the dense point cloud reconstruction process in weak texture areas, thereby significantly reducing the overall reconstruction time.

[0120] The method of determining the dense feature matching results in the embodiment of the present application can bypass the Euclidean distance process of calculating sparse feature descriptors and descriptor matching in traditional technologies, and directly obtain the dense feature matching results of the images of the two detection targets. In traditional feature matching technology, some structural points of the data are usually matched, so the reconstructed model only has sparse structural information. The feature matching effect is better for static detection targets such as buildings, but the feature matching effect is poor for organisms with smoother skin such as humans. Compared with traditional feature matching technology, the present application can determine the matching results of more weak texture areas, so that the three-dimensional point cloud information can be determined more accurately and completely.

[0121] In some embodiments, performing three-dimensional reconstruction based on the dense feature matching results and the epipolar constraint relationship to determine the overall three-dimensional point cloud information of the detection target includes:

[0122] S8051: Determine a basic matrix according to the dense feature matching result.

[0123] S8053: Determine an essential matrix based on the basic matrix, and perform singular value decomposition on the essential matrix to determine a camera pose matrix.

[0124] S8055: Determine the overall three-dimensional point cloud information based on the camera pose matrix and the epipolar constraint relationship.

[0125] In an embodiment of the present application, the optical center of the camera device when acquiring the image of the first frame detection target is used as the origin, and a spatial rectangular coordinate system is constructed to reconstruct the three-dimensional point cloud information. In some specific embodiments, a pinhole camera is used as a camera model to establish a target projection process. Among the images of multiple detection targets, an image containing more feature point matches and a frontal perspective covering more feature points of the detection target is selected as a key frame image. Specifically, the key frame image can be determined by calculating the matching quality of adjacent images, parallax change, and other indicators. A basic matrix is ​​constructed based on the key frame image and the dense feature matching results. The basic matrix is ​​used to describe the geometric relationship between the two images. The essential matrix can be determined based on the camera intrinsic parameters and the basic matrix. The essential matrix is ​​used to describe the relative posture relationship between the cameras. In the feature matching process, there may be erroneous feature matching points, which in turn lead to deviations in determining the camera pose matrix. Therefore, the RANSAC (random sampling consensus) method can be used to iteratively eliminate erroneous matching feature points in the process of solving the essential matrix, thereby improving the accuracy of subsequent determination of the three-dimensional point cloud information.

[0126] By performing singular value decomposition on the essential matrix, the camera pose matrix, that is, the camera rotation matrix and the camera translation matrix, can be determined. According to the camera pose matrix and the epipolar constraint relationship, the three-dimensional point cloud information is determined. Among them, the epipolar geometry describes how the corresponding feature points in the two images are mapped to each other. The position information of the three-dimensional point cloud can be determined more accurately through the epipolar constraint. After determining the camera pose matrix, the position information of the three-dimensional points of the corresponding points in the image of the detection target can be calculated by triangulation. For each pair of matching feature points, the position information of the three-dimensional points in the three-dimensional point cloud can be inferred using the camera pose matrix. By performing triangulation operations on multiple matching feature points, a dense three-dimensional point cloud model can be determined.

[0127] In order to further improve the accuracy of the overall three-dimensional point cloud information, after determining the overall three-dimensional point cloud information, it also includes removing outliers in the overall three-dimensional point cloud information and adaptively filtering the overall three-dimensional point cloud information; and smoothing the overall three-dimensional point cloud information according to the moving least squares method.

[0128] In the embodiments of the present application, denoising the overall 3D point cloud information can remove outliers from the overall 3D point cloud information; adaptive filtering of the overall 3D point cloud information can ensure that the accuracy of the original overall 3D point cloud information is not changed due to filtering; and the MLS (Moving Least Square) algorithm is executed on the cleaned overall 3D point cloud information to smooth the overall 3D point cloud information and improve the efficiency of determining the mesh model. Through the above optimization process of the overall 3D point cloud information, the density of the overall 3D point cloud information can be made more uniform.

[0129] The following describes a specific method for generating a grid model through an embodiment of the present application. In some embodiments, generating the grid model based on the overall three-dimensional point cloud information includes:

[0130] S901: Determine normal information of each point in the overall three-dimensional point cloud information, and correct the normal information according to a camera pose matrix, where the camera pose matrix is ​​determined according to the dense feature matching result.

[0131] S903: Projecting the entire three-dimensional point cloud information onto a two-dimensional plane according to the normal information to obtain a projected point cloud.

[0132] S905: Determine the topological connection relationship of each projection point in the projected point cloud according to triangulation;

[0133] S907: Generate the grid model according to the topological connection relationship.

[0134] In an embodiment of the present application, the normal information of each point in the overall three-dimensional point cloud information can be determined based on a neighboring point search algorithm. The neighboring point search algorithm may include algorithms such as a KD tree (K-Dimensional Tree) or an octree (Octree). According to the neighboring point search algorithm, the nearest points within a preset radius or a preset number of nearest points of each point in the overall three-dimensional point cloud information can be efficiently searched. A covariance matrix is ​​constructed for the neighboring point set of each three-dimensional point in the overall three-dimensional point cloud information, and the covariance matrix is ​​used to describe the local geometric distribution of the neighboring point set. Performing eigenvalue decomposition (EigenDecomposition) on the covariance matrix can obtain three eigenvalues ​​and their corresponding eigenvectors. The size of the eigenvalue indicates the degree of change in the local geometric morphology, and the direction of the eigenvector corresponding to the minimum eigenvalue is the normal information of the corresponding three-dimensional point.

[0135] Because the normals of 3D points in the overall 3D point cloud may face inward or outward, the normal information can be corrected based on the camera pose matrix. The camera pose matrix determines the direction of the camera observation vector, and the normal direction must be consistent with the camera observation vector. If the dot product of the normal information and the camera observation vector is negative, the normal direction points inward and needs to be flipped to the opposite direction, pointing outward. If the dot product of the normal information and the camera observation vector is positive, the normal direction points outward and no adjustment is required. After correcting the normal information, the overall 3D point cloud with the normal facing outward is obtained.

[0136] According to the normal information of each point in the overall three-dimensional point cloud information, the overall three-dimensional point cloud information is projected onto a two-dimensional plane to obtain a projected point cloud. Specifically, the projected point cloud can be a projection matrix. On the two-dimensional plane where the overall three-dimensional point cloud information is projected, the projected points are triangulated to generate a set of triangular facets, and the topological connection relationship between each projected point and its adjacent points is recorded. According to the topological connection relationship, the various points in the projected point cloud are connected to form triangular facets, and then the connection relationship between the points in the two-dimensional plane and the triangular facets is mapped back to the three-dimensional space. In the three-dimensional space, according to the topological connection relationship, the various three-dimensional points in the three-dimensional point cloud are connected to form triangular facets, thereby generating a three-dimensional mesh model (Mesh model) composed of triangles.

[0137] The following describes a method for determining the second three-dimensional point cloud information through an embodiment of the present application.

[0138] In some embodiments, a second local mesh model that matches a symmetrical portion of the portion to be inspected may be determined, and the second three-dimensional point cloud information corresponding to the second local mesh model may be determined. Specifically, a symmetrical portion symmetrical to the portion to be inspected may be determined within the mesh model, and the matching second three-dimensional point cloud information may be determined based on the mesh model of the symmetrical portion.

[0139] In other embodiments, the method of determining the second three-dimensional point cloud information may also include determining, in the overall three-dimensional point cloud information, the point cloud information closest to each three-dimensional point of the mirrored three-dimensional point cloud information based on the Euclidean distance, as the second three-dimensional point cloud information.

[0140] The following describes a specific method for determining the point cloud information matching result through an embodiment of the present application. In some embodiments, determining the point cloud information matching result between the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information includes:

[0141] S1001: Construct a transformation matrix according to each three-dimensional point and the corresponding mirror image three-dimensional point in the second three-dimensional point cloud information.

[0142] S1003: Iteratively updating the mirrored three-dimensional point cloud information according to an iterative closest point algorithm and the transformation matrix, and determining a point cloud information matching result according to the updated mirrored three-dimensional point cloud information and the second three-dimensional point cloud information.

[0143] In this embodiment, after determining the mirrored 3D point cloud information, the mirrored 3D point closest to each 3D point in the second 3D point cloud information is determined in the mirrored 3D point cloud information based on Euclidean distance. Based on each pair of closest 3D points, an essential matrix is ​​constructed and singular value decomposition is performed to determine a transformation matrix. The transformation matrix includes a rotation matrix and a translation vector.

[0144] Since the positive direction of the z-axis determined by the principal component analysis (PCA) algorithm may have deviations, the mirrored 3D point cloud information may not be completely left-right symmetrical with the first 3D point cloud information, and the zy plane of the local point cloud coordinate system may not be a mirror symmetry plane, it is necessary to use the iterative closest point (ICP) algorithm to align the second 3D point cloud information with the mirrored point cloud information.

[0145] The yz plane is generated based on the y-axis and z-axis of the local point cloud coordinate system. The mirror plane can be obtained by applying the yz plane to the homogeneous matrix aligned by the Iterative Closest Point (ICP) algorithm. The mirror plane can be displayed transparently in the Mesh model. Repeatedly determine the mirrored 3D point closest to each 3D point in the second 3D point cloud information and determine the transformation matrix until the Iterative Closest Point (ICP) algorithm converges. The closest 3D point pair in the second 3D point cloud information and the mirrored 3D point cloud information determined by the last iteration is the point cloud information matching result.

[0146] The following describes several specific embodiments of displaying the symmetry detection results using preset colors when the x-axis, y-axis, and z-axis are selected individually or when the x-axis, y-axis, and z-axis are selected simultaneously.

[0147] If the x-axis difference is selected, assume the mirrored 3D point P in the mirrored 3D point cloud and the corresponding point Q in the second 3D point cloud. Calculate the x-axis difference between P and Q: dx = Px - Qx. Map the difference to a pixel value C between 0 and 255: C = abs(dx) / (2*T)*255, where T is the preset precision and abs represents the absolute value. Set an upper limit for the pixel value C. If C exceeds 255 (i.e., the distance difference exceeds 2T, which can be customized to 3mm, for example), set the pixel value C to 255. If dx > 0, indicating that the mirrored 3D point corresponds to the point in the second 3D point cloud in the positive x-axis direction, set the color of P to red (C, 0, 0). The larger the absolute value of abs(C), the redder the point. If dx < 0, indicating that the mirrored 3D point corresponds to the point in the second 3D point cloud in the negative x-axis direction, set the color of P to cyan (0, C, C). The larger the absolute value of abs(C), the cyanier the point.

[0148] If the difference in the y-axis is selected, assume the mirrored 3D point P in the mirrored 3D point cloud and the corresponding point Q in the second 3D point cloud. Calculate the y-axis difference between P and Q: dy = Py - Qy. Map the difference to a pixel value C between 0 and 255: C = abs(dy) / (2*T)*255, where T is the preset precision value and abs represents the absolute value. Set an upper limit for the pixel value C. If C exceeds 255 (i.e., the distance difference exceeds 2T, which can be customized to 3mm, for example), set the pixel value C to 255. If dy > 0, indicating that the mirrored 3D point corresponds to the point in the second 3D point cloud in the positive y-axis direction, set the color of P to green (0, C, 0). The larger the absolute value of abs(C), the greener the point. If dy < 0, indicating that the mirrored 3D point corresponds to the point in the second 3D point cloud in the negative y-axis direction, set the color of P to magenta (C, 0, C). The larger the absolute value of abs(C), the magenta the point.

[0149] If the z-axis difference is selected, assume the mirrored 3D point P in the mirrored 3D point cloud and the corresponding point Q in the second 3D point cloud. Calculate the z-axis difference between P and Q: dz = Pz - Qz. Map the difference to a pixel value C between 0 and 255: C = abs(dz) / (2*T)*255, where T is the preset precision and abs represents the absolute value. Set an upper limit for the pixel value C. If C exceeds 255 (i.e., the distance difference exceeds 2T, which can be customized to 3mm, for example), set the pixel value C to 255. If dz > 0, indicating that the mirrored 3D point corresponds to the point in the second 3D point cloud in the positive z-axis direction, set the color of P to blue (0, 0, C). The larger the absolute value of abs(C), the bluer the point. If dz < 0, indicating that the mirrored 3D point corresponds to the point in the second 3D point cloud in the negative z-axis direction, set the color of P to yellow (C, C, 0). The larger the absolute value of abs(C), the yellower the point.

[0150] If you select differences along the x, y, and z axes, assume the mirrored 3D point P in the mirrored 3D point cloud and its corresponding point Q in the second 3D point cloud. Calculate the differences between P and Q along the x, y, and z axes: dx = Px - Qx, dy = Py - Qy, and dz = Pz - Qz. Map the differences dx, dy, and dz to pixel values ​​cx, cy, and cz of (-127, 127), respectively: cx = dx / (2*T)*127, cy = dy / (2*T)*127, and cz = dz / (2*T)*127, where T is the preset precision value. Set an upper limit for the pixel value: if the pixel value exceeds 127 (i.e., the distance difference exceeds 2T, which can be customized to 3mm, for example), the pixel value is set to 127; if the pixel value is less than -127, the pixel value is set to -127. Set the color of P to gray (127+cx, 127+cy, 127+cz). When the Euclidean distance between the mirrored 3D point in the mirrored 3D point cloud information and the corresponding 3D point in the second 3D point cloud information is close, the color will be gray. If there is a large difference in a certain direction, the corresponding color will change.

[0151] Traditionally, medical imaging data can be reconstructed and body structure determined through 3D reconstruction techniques using medical imaging data from CT and MRI. However, these techniques require specialized medical image processing software and cannot be applied in real time to symmetry detection, nor can they provide real-time guidance to professionals. While 3D surface reconstruction using image data can reconstruct the body's external structure, it typically involves delayed quantitative assessment and cannot provide real-time guidance to professionals. Furthermore, specialized processing equipment is required and assessment can only be performed on specific areas, making it inapplicable to all.

[0152] In order to further improve the accuracy of the grid model in describing the detection target, this application also provides two methods for further optimizing the grid model.

[0153] In some embodiments, the mesh model includes a triangular mesh model, and after generating the mesh model according to the overall three-dimensional point cloud information, the method further includes:

[0154] S1101: Determine a first average value of lengths of all boundary edges in the triangular mesh model, where the boundary edges include edges connecting only one triangle in the triangular mesh model.

[0155] S1103: Determine the distance between two adjacent boundary points of a minimum angle boundary point, where the minimum angle boundary point includes, among the boundary points of the triangular mesh model, a boundary point where the angle between two connected boundary edges is the smallest;

[0156] S1105: If the distance is less than a first threshold, connecting two adjacent boundary points of the minimum angle boundary point, where the first threshold is determined according to the first average value.

[0157] S1207: If the distance is greater than or equal to the first threshold, connect the two adjacent boundary points of the minimum angle boundary point, determine the midpoint of the boundary edge obtained after connecting the two adjacent boundary points, and connect the minimum angle boundary point and the midpoint.

[0158] In the embodiments of the present application, holes may exist in the mesh model determined based on 3D point cloud information. When traversing the edges of the mesh model, if an edge connects only one triangle, this edge is called a mesh boundary edge. If the boundary edges in the boundary edge subset can be sequentially connected to form a ring, these connected boundary edges form holes in the mesh. The holes found can be repaired sequentially.

[0159] Calculate the first average value D of the lengths of all boundary edges in the triangular mesh model. Determine the minimum angle boundary point A with the minimum angle, and calculate the point spacing S between the two adjacent boundary points B and C of A. If the spacing is less than the first threshold, connect the two adjacent boundary points B and C and add a triangle ABC; if it is greater than the first threshold, after connecting the two adjacent boundary points B and C, add two triangles ABM and AMC at the midpoint M of the BC side and point A. The first threshold can be twice the first average value D. In some specific embodiments, the schematic diagram of filling holes in the mesh model when the spacing is less than the first threshold is as follows Figure 3 As shown, Figure 3 Midpoint V i-1 With point V i+1 In other specific embodiments, the schematic diagram of filling holes in the grid model when the spacing is greater than or equal to the first threshold is as follows: Figure 4 As shown, Figure 4 Midpoint V i-1 With point V i+1 are respectively point B, point C, and V in the above embodiment. new Connect two adjacent boundary points to get the midpoint of the boundary edge.

[0160] In some embodiments, the mesh model may be further homogenized. The mesh model includes a triangular mesh model, and after generating the mesh model based on the overall three-dimensional point cloud information, the following steps may be further included:

[0161] S1201: Determine a second average value of the lengths of all triangle edges in the triangular mesh model.

[0162] S1203: If the triangle side to be measured is smaller than a second threshold, deleting the triangle side to be measured and merging two endpoints of the triangle side to be measured into a midpoint of the triangle side, wherein the second threshold is determined according to the second average value.

[0163] S1205: If the side of the triangle to be measured is greater than a third threshold, control all vertices in the two triangles connected by the side of the triangle to be measured to connect with the midpoint of the side of the triangle to be measured, and the third threshold is determined according to the second average value, and the third threshold is greater than the second threshold.

[0164] In the embodiment of the present application, the mesh model is homogenized by eliminating edges that are too long or too short, reducing narrow and long triangles, too small triangles, and too large triangles.

[0165] Determine a second average value, L, of the lengths of all triangle edges in the triangular mesh model. Each edge is evaluated sequentially. If the length of an edge is less than a second threshold, the edge and the two triangles associated with the edge are deleted, and the two vertices at the opposite ends of the edge are merged into a single vertex, with the vertex position being the midpoint of the two original endpoints. If the length of an edge is greater than the second threshold, perform a vertex addition operation, adding a new vertex at the midpoint of the edge and splitting the edge into two edges. The two triangles associated with the original edge are then combined to form four new triangles.

[0166] The following describes the symmetry detection method provided by this application through a specific application scenario.

[0167] A newborn's skull hasn't fully hardened and solidified for a period of time after birth. Prolonged side-sleeping can cause a change in head shape, a condition known as torticollis. This condition can affect facial symmetry, making the left and right sides of the face appear unequal. The symmetry test provided in this application can analyze a child's face for symmetry.

[0168] Using a mobile phone, facial images of the newborn are taken from different angles, and feature points are matched to reconstruct a dense 3D point cloud model of the face. The 3D point cloud model is then cleaned through denoising and filtering to create a smooth, uniformly dense 3D point cloud model. This 3D point cloud model is then reconstructed into a mesh data model and displayed on the phone screen. Parents interact with the phone screen to circle the facial area that needs to be evaluated. A local point cloud coordinate system is established for the selected area using the phone's IMU information, and the 3D point cloud information is mirrored to determine the point-pair matching relationship between the mirrored point cloud information and the original point cloud. Based on the difference in point pairs in different directions in the local point cloud coordinate system, the mirrored point cloud is assigned color information, and the colored point cloud data is superimposed on the original facial mesh model to intuitively display any symmetry differences.

[0169] In an embodiment of the present application, a visual symmetry detection method is also provided, including:

[0170] S1301: In response to a selection instruction of an input device, determining a portion of a detection target to be detected.

[0171] S1303: Determine first three-dimensional point cloud information according to the part to be detected.

[0172] S1305: Obtain second three-dimensional point cloud information based on the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information.

[0173] S1307: Displaying the symmetry detection result through a display device; the symmetry detection result is determined based on the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information and the second three-dimensional point cloud information.

[0174] In an embodiment of the present application, the detection target may include a biological target to be detected, such as a human body, an animal body, a plant body, etc. The part to be detected may correspond to a certain part or the entirety of the detection target, or may correspond to multiple parts of the detection target. In some embodiments, the part to be detected of the detection target may be determined in response to a selection instruction of an input device. Specifically, a doctor or an evaluator may circle or select the part to be detected of the detection target according to a preset option through an input device, for example, circling the human arm as the part to be detected, or selecting a preset facial option as the part to be detected, etc.

[0175] In the embodiment of the present application, after determining the part to be detected and the symmetrical region of the part to be detected, first three-dimensional point cloud information matching the part to be detected can be determined based on the mapping relationship between the mesh model of the detection target and the three-dimensional point cloud information. Based on the first three-dimensional point cloud information corresponding to the part to be detected, a mirror image of the first three-dimensional point cloud information can be determined through a mirror transformation, and then the marked second three-dimensional point cloud information can be obtained based on the mirror image of the first three-dimensional point cloud information.

[0176] Finally, based on the mirror image point cloud data of the first point cloud data and the second point cloud data, the symmetry detection result is determined, and the symmetry detection result is displayed through a display device. Specifically, the symmetry detection results of different parts can be displayed by different visual distinction marks. The specific method for determining the symmetry detection result and the method for visual distinction marks refer to the aforementioned embodiment and will not be repeated here. In addition, the method for determining the first three-dimensional point cloud information, the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information, and the second three-dimensional point cloud information in the embodiment of the present application can all refer to the aforementioned embodiment and will not be repeated here.

[0177] The symmetry detection method provided in the embodiment of the present application determines the part to be detected of the detection target in response to the selection instruction of the input device, determines the first three-dimensional point cloud information based on the part to be detected, and then determines the second three-dimensional point cloud information, and can visually display the symmetry detection results.

[0178] On the other hand, since the part to be inspected can be interactively adjusted in real time, the present application can also display the symmetrical part inspection results in real time and interactively during the process of changes in the part to be inspected, which is efficient and convenient.

[0179] In some embodiments, after adjusting the mirrored three-dimensional point, the adjusted symmetry detection result can be displayed in real time. After displaying the symmetry detection result on the display device, the method further includes:

[0180] S1401: In response to a movement instruction of an input device, after moving the mirrored three-dimensional point, updated mirrored three-dimensional point cloud information is obtained.

[0181] S1403: Determine an updated coordinate difference value in the direction of the target coordinate axis between the updated mirrored three-dimensional point cloud information and the corresponding three-dimensional point in the second three-dimensional point cloud information.

[0182] S1405: Displaying the updated symmetry detection result of the part to be detected via the display device according to the updated coordinate difference value and the updated visual distinguishing mark.

[0183] In the embodiment of the present application, the symmetry test results of the inspected part can be evaluated and corrected. Specifically, the evaluator can move the mirror point cloud plane or mirror point through the input device, and after moving the mirror 3D point, the updated mirror 3D point cloud information is obtained.

[0184] Determine an updated coordinate difference between the updated mirrored three-dimensional point cloud information and the corresponding three-dimensional point in the second three-dimensional point cloud information in the direction of the target coordinate axis. Then, based on the updated coordinate difference and the updated visual distinguishing mark, determine an updated symmetry test result for the part to be tested in real time via the display device.

[0185] After the symmetry detection results are determined, the present application can recalculate the matching relationship, correct and re-detect the symmetry in real time, and determine the updated symmetry detection results in real time based on the re-assigned visual distinguishing marks. It is understandable that symmetry detection may deviate from the actual state of the detection target, the expected state, or the experience of the evaluator. In this case, the part to be detected can be fine-tuned or gradually adjusted to determine the adjusted symmetry detection results in real time, effectively improving the real-time and detection efficiency of symmetry detection.

[0186] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0187] Based on the same inventive concept, embodiments of the present application further provide a symmetry detection device 1500 for implementing the aforementioned symmetry detection method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more symmetry detection device embodiments provided below can be found in the above-described limitations of the symmetry detection method and will not be further elaborated here.

[0188] In one embodiment, Figure 5 As shown, a symmetry detection device 1500 is provided, comprising:

[0189] A mesh model acquisition module 1501 is configured to acquire a mesh model of a detection target, wherein the mesh model includes first three-dimensional point cloud information matching a part to be detected in the detection target;

[0190] A mirrored 3D point cloud information generating module 1502 is configured to generate mirrored 3D point cloud information based on the first 3D point cloud information;

[0191] A second three-dimensional point cloud information generating module 1503 is configured to determine, based on the mirrored three-dimensional point cloud information, second three-dimensional point cloud information that matches a symmetrical portion of the portion to be detected;

[0192] The symmetry detection module 1504 is configured to determine a symmetry detection result of the part to be detected based on a matching result between the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information.

[0193] In some embodiments, the mirrored three-dimensional point cloud information generation module 1502 is also used to obtain the posture information of the detection target; determine the local point cloud coordinate system based on the first three-dimensional point cloud information and the posture information; determine the mirror coordinates of the first three-dimensional point cloud information coordinates according to the x-axis direction of the local point cloud coordinate system, and determine the mirrored three-dimensional point cloud information based on the mirror coordinates.

[0194] In some embodiments, the mirrored three-dimensional point cloud information generation module 1502 is also used to determine the origin of the local point cloud coordinate system based on the centroid point cloud information in the first three-dimensional point cloud information; determine the z-axis direction of the local point cloud coordinate system based on the direction of the first three-dimensional point cloud information; determine the projection component perpendicular to the ground projected onto the three-dimensional point cloud information plane based on the posture information, and determine the projection component perpendicular to the z-axis as the x-axis direction of the local point cloud coordinate system; determine the y-axis direction based on the perpendicular direction of the plane formed by the x-axis and the z-axis; and obtain the local point cloud coordinate system based on the origin, x-axis direction, y-axis direction, and z-axis direction.

[0195] In some embodiments, the mirrored three-dimensional point cloud information generation module 1502 is also used to determine the main direction of the first three-dimensional point cloud information as the positive direction of the z-axis of the local point cloud coordinate system; determine the positive direction of the y-axis of the local point cloud coordinate system, and the angle between the positive direction of the y-axis and the vertical direction of the ground is less than 90 degrees; determine the positive direction of the x-axis based on the positive direction of the z-axis and the positive direction of the y-axis, and determine the local point cloud coordinate system based on the positive direction of the x-axis, the positive direction of the y-axis and the positive direction of the z-axis.

[0196] In some embodiments, the symmetry detection module 1504 is also used to determine the coordinate difference value in the direction of the target coordinate axis between the mirrored three-dimensional point in the mirrored three-dimensional point cloud information and the corresponding three-dimensional point in the second three-dimensional point cloud information based on the matching result and the preset accuracy value; and determine the symmetry detection result of the part to be detected in the grid model based on the coordinate difference value and the visual distinguishing mark.

[0197] In some embodiments, the visual distinguishing mark includes a preset color, and the symmetry detection module 1504 is also used to set the corresponding mirrored three-dimensional point to the first color if the coordinate difference value is greater than 0, and the size of the difference value is positively correlated with the depth of the first color; if the coordinate difference value is less than 0, the corresponding mirrored three-dimensional point is set to the second color, and the size of the difference value is negatively correlated with the depth of the second color; the mirrored three-dimensional point after setting the first color or the second color is superimposed on the grid model, and the symmetry detection result is determined based on the superimposed grid model.

[0198] The embodiment of the present application further provides a symmetry detection device 1600 for implementing the above-mentioned symmetry detection method. Figure 6 The solution provided by the device to solve the problem is similar to the solution described in the above method, so the specific limitations of one or more symmetry detection device embodiments provided below can refer to the limitations of the symmetry detection method above and will not be repeated here.

[0199] In one embodiment, a symmetry detection device 1600 is provided, comprising:

[0200] The to-be-detected part determination module 1601 is configured to determine the to-be-detected part of the detection target in response to a selection instruction of an input device;

[0201] A first three-dimensional point cloud information determining module 1602 is configured to determine first three-dimensional point cloud information based on the part to be detected;

[0202] A second three-dimensional point cloud information determination module 1603 is configured to obtain second three-dimensional point cloud information based on the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information;

[0203] The symmetry detection result display module 1604 is used to display the symmetry detection result through a display device; the symmetry detection result is determined based on the mirror image 3D point cloud information of the first 3D point cloud information and the second 3D point cloud information.

[0204] In some embodiments, the symmetry detection device 1600 is also used to respond to the movement instruction of the input device, move the mirrored three-dimensional point, and obtain updated mirrored three-dimensional point cloud information; determine the updated coordinate difference value between the updated mirrored three-dimensional point cloud information and the corresponding three-dimensional point in the second three-dimensional point cloud information in the direction of the target coordinate axis; and display the updated symmetry detection result of the part to be detected through the display device based on the updated coordinate difference value and the updated visual distinguishing mark.

[0205] Each module in the symmetry detection apparatus 1500 and the symmetry detection apparatus 1600 can be implemented in whole or in part via software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0206] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a symmetry detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0207] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0208] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the symmetry detection method described in any of the above embodiments are implemented.

[0209] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the symmetry detection method described in any one of the above embodiments.

[0210] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0211] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0212] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0213] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0214] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0215] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A symmetry detection method, characterized in that: The method comprises: Acquire a grid model of a detection target, wherein the grid model includes first three-dimensional point cloud information matching a part to be detected in the detection target; generating mirrored three-dimensional point cloud information based on the first three-dimensional point cloud information; Determining, based on the mirrored three-dimensional point cloud information, second three-dimensional point cloud information that matches a symmetrical portion of the portion to be detected; A symmetry detection result of the part to be detected is determined according to a matching result between the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information.

2. The method according to claim 1, characterized in that Generating mirrored three-dimensional point cloud information according to the first three-dimensional point cloud information includes: Acquiring posture information of the detection target; Determine a local point cloud coordinate system according to the first three-dimensional point cloud information and the posture information; According to the x-axis direction of the local point cloud coordinate system, the mirror coordinates of the first three-dimensional point cloud information coordinates are determined, and the mirror three-dimensional point cloud information is determined according to the mirror coordinates.

3. The method according to claim 2, characterized in that Determining a local point cloud coordinate system according to the first three-dimensional point cloud information and the posture information includes: Determining the origin of the local point cloud coordinate system according to the centroid point cloud information in the first three-dimensional point cloud information; determining a z-axis direction of the local point cloud coordinate system according to the direction of the first three-dimensional point cloud information; Determine, based on the posture information, a projection component perpendicular to the ground projected onto the three-dimensional point cloud information plane, and determine the projection component perpendicular to the z-axis as the x-axis direction of the local point cloud coordinate system; Determine the y-axis direction based on the perpendicular direction of the plane formed by the x-axis and the z-axis; The local point cloud coordinate system is obtained according to the origin, x-axis direction, y-axis direction, and z-axis direction.

4. The method according to claim 3, characterized in that Obtaining the local point cloud coordinate system according to the origin, the x-axis direction, the y-axis direction, and the z-axis direction includes: Determine a main direction of the first three-dimensional point cloud information as the positive z-axis direction of the local point cloud coordinate system; Determine the positive direction of the y-axis of the local point cloud coordinate system, where the angle between the positive direction of the y-axis and the vertical direction of the ground is less than 90 degrees; The positive direction of the x-axis is determined according to the positive direction of the z-axis and the positive direction of the y-axis, and the local point cloud coordinate system is determined according to the positive direction of the x-axis, the positive direction of the y-axis, and the positive direction of the z-axis.

5. The method according to claim 1, characterized in that Determining the symmetry detection result of the to-be-detected portion according to the matching result of the mirrored three-dimensional point cloud information and the second three-dimensional point cloud information includes: Determining, based on the matching result and a preset accuracy value, a coordinate difference value between a mirrored 3D point in the mirrored 3D point cloud information and a corresponding 3D point in the second 3D point cloud information in a target coordinate axis direction; A symmetry detection result of the part to be detected is determined in the grid model according to the coordinate difference value and the visual distinguishing mark.

6. The method according to claim 5, characterized in that The visual distinguishing mark includes a preset color, and determining the symmetry detection result of the to-be-detected portion in the grid model according to the coordinate difference value and the visual distinguishing mark includes: If the coordinate difference value is greater than 0, the corresponding mirrored three-dimensional point is set to the first color, and the size of the difference value is positively correlated with the depth of the first color; If the coordinate difference value is less than 0, the corresponding mirrored 3D point is set to a second color, and the size of the difference value is negatively correlated with the depth of the second color; The mirrored three-dimensional points after being set with the first color or the second color are superimposed on the grid model, and the symmetry detection result is determined based on the superimposed grid model.

7. A symmetry detection method, characterized in that: The method comprises: In response to a selection instruction of an input device, determining a portion of a detection target to be detected; Determining first three-dimensional point cloud information according to the part to be detected; Obtaining second three-dimensional point cloud information based on the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information; The symmetry detection result is displayed through a display device; the symmetry detection result is determined according to the mirror image three-dimensional point cloud information of the first three-dimensional point cloud information and the second three-dimensional point cloud information.

8. The method according to claim 7, characterized in that After displaying the symmetry detection result on the display device, the method further includes: In response to a movement instruction of the input device, the mirrored three-dimensional point is moved to obtain updated mirrored three-dimensional point cloud information; Determining an updated coordinate difference value in the direction of the target coordinate axis between the updated mirrored three-dimensional point cloud information and the corresponding three-dimensional point in the second three-dimensional point cloud information; The updated symmetry detection result of the part to be detected is displayed on the display device according to the updated coordinate difference value and the updated visual distinguishing mark.

9. A symmetry detection device, characterized in that: The device comprises: A grid model acquisition module, configured to acquire a grid model of a detection target, wherein the grid model includes first three-dimensional point cloud information matching a part to be detected in the detection target; a mirrored three-dimensional point cloud information generating module, configured to generate mirrored three-dimensional point cloud information based on the first three-dimensional point cloud information; A second three-dimensional point cloud information generating module is used to determine second three-dimensional point cloud information that matches a symmetrical portion of the portion to be detected based on the mirrored three-dimensional point cloud information; The symmetry detection module is used to determine the symmetry detection result of the part to be detected based on the matching result of the mirror three-dimensional point cloud information and the second three-dimensional point cloud information.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images