Folding angle detection method, device and readable storage medium of foldable device

Abstract

The application relates to the technical field of image processing, and provides a folding angle detection method and device of a foldable equipment and a readable storage medium, which comprises the following steps: acquiring point cloud diagrams of first and second middle frames in a flat state. After converting three-dimensional coordinates of point clouds in a first middle frame region in the point cloud diagram into two-dimensional coordinates of the point clouds, a straight line fitting is performed on two-dimensional coordinates of point clouds of a pixel row in the first middle frame region to obtain a first fitting straight line. After converting three-dimensional coordinates of point clouds in a second middle frame region in the point cloud diagram into two-dimensional coordinates of the point clouds, a straight line fitting is performed on two-dimensional coordinates of point clouds of a pixel row in the second middle frame region to obtain a second fitting straight line. Finally, the inclination angles of the first fitting straight line and the second fitting straight line with a reference plane are calculated, and the folding angles of the first and second middle frames are determined according to the two inclination angles. Therefore, the method can realize the detection of the folding angles of the two middle frames of the foldable equipment through the point cloud diagram, so that the detection accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device, and readable storage medium for detecting the flattening angle of a foldable device. Background Technology

[0002] With the development of electronic device technology, various foldable devices have emerged. Currently, foldable devices can achieve the required functions based on the folding angle. For example, foldable devices can switch displays based on the folding angle. Therefore, how to accurately detect the folding angle of a foldable device in its flattened state is a problem that urgently needs to be solved. Summary of the Invention

[0003] This application provides a method, device, and readable storage medium for detecting the flattened angle of a foldable device, which can accurately detect the included angle (i.e., flattened angle) of the foldable device in the flattened state to ensure the accuracy of subsequent function implementation.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] Firstly, a method for detecting the flattening angle of a foldable device is provided, the method comprising:

[0006] Obtain point cloud images of the first and second middle frames in a flattened state. The point cloud images include a first middle frame region corresponding to the first middle frame and a second middle frame region corresponding to the second middle frame. After converting the three-dimensional coordinates of the point cloud in the first middle frame region into two-dimensional coordinates, determine the two-dimensional coordinates of a pixel row based on the two-dimensional coordinates of the pixel rows and columns in the first middle frame region and perform line fitting to obtain a first fitted line. After converting the three-dimensional coordinates of the point cloud in the second middle frame region into two-dimensional coordinates, determine the two-dimensional coordinates of a pixel row based on the two-dimensional coordinates of the pixel rows and columns in the second middle frame region and perform line fitting to obtain a second fitted line. Calculate the first tilt angle between the first fitted line and the reference plane and calculate the second tilt angle between the second fitted line and the reference plane. Determine the flattened angle of the first and second middle frames based on the first and second tilt angles.

[0007] Therefore, this method, after obtaining the point cloud images of the first and second middle frames in their flattened state, first obtains the cross-sectional lines (i.e., the first and second fitted straight lines) of the first and second middle frames by converting the point cloud data from 3D to 2D and fitting surfaces to straight lines. Then, it directly detects the flattened angles of the first and second middle frames based on the tilt angle between the cross-sectional lines and a reference plane (such as a horizontal plane). This method is more direct than the traditional method of indirectly representing flattened angles through the angle formed by spatial normal vectors, thus improving the accuracy of flattened angle detection.

[0008] In one possible implementation of the first aspect, acquiring point cloud images of the first and second mid-frames in a flattened state includes: receiving point cloud images transmitted by a spectral imager, wherein the point cloud images are obtained by the spectral imager based on the measured depth maps and two-dimensional spatial information of the first and second mid-frames in a flattened state. In this implementation, acquiring the point cloud images via a spectral imager avoids the influence of factors such as reflections on point cloud imaging compared to methods such as lidar.

[0009] In one possible implementation of the first aspect, in order to ensure that the ratio of the first and second inner frames in the point cloud map matches the actual ratio, so as to avoid affecting the detection of the flattened angle, the method may further include: reducing or enlarging the point cloud map.

[0010] In one possible implementation of the first aspect, scaling down or zooming in on the point cloud map may include:

[0011] The scanning speed during point cloud imaging is obtained, the calibration scale of the preset calibration plate is set, and the z-coordinate transformation threshold is set; the z-coordinate transformation threshold is given by the spectral imager. The point cloud image is scaled down or enlarged according to the scanning speed, calibration scale, and z-coordinate transformation threshold. The point cloud image includes multiple point cloud three-dimensional coordinates. The product of the x-coordinate of the point cloud three-dimensional coordinates and the scanning speed is the scaled-down or enlarged x-coordinate. The product of the y-coordinate of the point cloud three-dimensional coordinates and the calibration scale is the scaled-down or enlarged y-coordinate. The product of the z-coordinate of the point cloud three-dimensional coordinates and the z-coordinate transformation threshold is the scaled-down or enlarged z-coordinate. The point cloud three-dimensional coordinates include the first point cloud three-dimensional coordinates and the second point cloud three-dimensional coordinates.

[0012] In one possible implementation of the first aspect, a first fitted line and a second fitted line can be obtained by randomly selecting a pixel row from each pixel row within the first and second frame regions and performing line fitting on the point cloud 2D coordinates. That is, a first fitted line can be obtained by arbitrarily selecting a pixel row within the first frame region and performing line fitting on the point cloud 2D coordinates within that selected pixel row. Similarly, a second fitted line can be obtained by arbitrarily selecting a pixel row within the second frame region and performing line fitting on the point cloud 2D coordinates within that selected pixel row.

[0013] In another possible implementation of the first aspect, considering the problem of inaccurate line fitting caused by pixel loss or anomalies due to uneven surfaces of the first and second middle frames, a point cloud 2D coordinate of a pixel row in the middle frame region can be determined by a fusion method of summing the y-coordinates by pixel column to perform line fitting.

[0014] Based on this, by determining the two-dimensional coordinates of the point cloud corresponding to the rows and columns of pixels within the first frame area, and performing line fitting on the two-dimensional coordinates of the point cloud of a pixel row to obtain the first fitted line, the following can be included:

[0015] After converting the 3D coordinates of the point cloud within the first frame region into 2D coordinates, the first mean coordinates corresponding to each pixel column within the first frame region are determined. The x-coordinate in the first mean coordinates is the x-coordinate of each point cloud 2D coordinate within the corresponding pixel column, and the y-coordinate is obtained by summing and averaging the y-coordinates of each point cloud 2D coordinate within the corresponding pixel column. A first fitted line is obtained by fitting the first mean coordinates corresponding to each pixel column within the first frame region. The 2D coordinates of a determined pixel row are the first mean coordinates corresponding to each pixel column within the first frame region.

[0016] In one possible implementation of the first aspect, based on the two-dimensional coordinates of the point cloud corresponding to the rows and columns of pixels within the second frame region, the two-dimensional coordinates of the point cloud of a pixel row are determined, and a second fitted line is obtained by line fitting. This may include:

[0017] After converting the 3D coordinates of the point cloud within the second frame region into 2D coordinates, the second mean coordinates corresponding to each pixel column within the second frame region are determined. The x-coordinate in the second mean coordinates is the x-coordinate of each point cloud 2D coordinate within the corresponding pixel column, and the y-coordinate is obtained by summing and averaging the y-coordinates of each point cloud 2D coordinate within the corresponding pixel column. A second fitted line is obtained by fitting the second mean coordinates corresponding to each pixel column within the second frame region. The 2D coordinates of a determined pixel row are the second mean coordinates corresponding to each pixel column within the second frame region.

[0018] In one possible implementation of the first aspect, to reduce the amount of data for flattening angle detection, at least one region of interest can be selected from the middle frame region, and then a straight line fitting can be performed based on the selected region of interest. Therefore, after converting the 3D coordinates of the point cloud within the first middle frame region to 2D coordinates, the first mean coordinates corresponding to each pixel column within the first middle frame region are determined; a straight line fitting is then performed on the first mean coordinates corresponding to each pixel column within the first middle frame region to obtain a first fitted straight line, which may include:

[0019] At least one region of interest (ROI) is selected within the first frame region. The 3D coordinates of the first point cloud within each ROI are converted into 2D coordinates. The x-coordinate in the 2D coordinates is the same as the x-coordinate in the 3D coordinates, and the y-coordinate is the same as the z-coordinate. For the 2D coordinates, the y-coordinates are summed column-wise according to the corresponding pixel columns in the ROI, and the average value is taken while keeping the x-coordinate unchanged, resulting in n first mean coordinates. Here, n is the number of pixel columns in the ROI, and n is a positive integer. The first mean coordinates corresponding to at least one ROI are then fitted with a straight line using the least squares method to obtain the first fitted straight line corresponding to at least one ROI.

[0020] In one possible implementation of the first aspect, after converting the three-dimensional coordinates of the point cloud within the second frame region into two-dimensional coordinates, the second mean coordinates corresponding to each pixel column within the second frame region are determined; a second fitted line is obtained by fitting the second mean coordinates corresponding to each pixel column within the second frame region, which may include:

[0021] At least one second region of interest (ROI) is selected within the second frame region. The 3D coordinates of the second point cloud within each ROI are converted into 2D coordinates. The x-coordinate in the 2D coordinates is the same as the x-coordinate in the 3D coordinates, and the y-coordinate is the same as the z-coordinate. For the 2D coordinates, the y-coordinate is summed column-wise according to the pixel column of the corresponding ROI, and the average value is taken while keeping the x-coordinate unchanged, resulting in m second mean coordinates. Here, m is the number of pixel columns in the ROI, and m is a positive integer. The second mean coordinates corresponding to at least one ROI are then fitted with a line using the least squares method to obtain the second fitted line corresponding to at least one ROI.

[0022] In one possible implementation of the first aspect, the selection of the region of interest can be specified according to actual needs. Therefore, selecting at least one first region of interest within the first frame region can include: randomly selecting at least one first region of interest from the first frame region according to a preset first region range, wherein the size of the first region of interest corresponds to the preset region size.

[0023] In one possible implementation of the first aspect, selecting at least one second region of interest within the second middle frame region may also include: randomly selecting at least one second region of interest from the second middle frame region according to a preset second region range, wherein the size of the second region of interest corresponds to the preset region size.

[0024] In one possible implementation of the first aspect, the tilt angle of the fitted line can be calculated using inverse trigonometric functions. Therefore, calculating the first tilt angle between the first fitted line and the reference plane, and calculating the second tilt angle between the second fitted line and the reference plane, and determining the flattening angle of the first and second middle frames based on the first and second tilt angles, can include:

[0025] Determine the starting and ending coordinates of the first fitted line, and use the inverse trigonometric function atan2 to calculate the first tilt angle between the first fitted line and the reference plane. Determine the starting and ending coordinates of the second fitted line, and use the inverse trigonometric function atan2 to calculate the second tilt angle between the second fitted line and the reference plane. Calculate the difference between the first tilt angle corresponding to the first fitted line and the second tilt angle corresponding to the second fitted line to obtain the flattening angle of the first and second middle frames in the flattened state.

[0026] In one possible implementation of the first aspect, after obtaining the flattening angle, the method may further include: outputting flattening angle indication information. The output flattening angle indication information may include:

[0027] Output the size information of the flattened angle; and / or, if the flattened angle is not within the preset angle range, determine that the flattened state of the first and second middle frames is unqualified and output a prompt message. This is used to prompt the user that there may be a problem with the assembly of the first and second middle frames, thereby avoiding the problem of unqualified flattened state caused by assembly errors.

[0028] In one possible implementation of the first aspect, to prevent interference from missing pixels and noise points, the image can be further smoothed using a smoothing filter. Based on this, the method may further include: after converting the three-dimensional coordinates of the points in the first and second inner frame regions of the point cloud image into two-dimensional coordinates, performing a smoothing filter on the first and second inner frame regions of the point cloud image.

[0029] Secondly, this application provides a flattening angle detection device for a foldable device, comprising: one or more processors and a memory, the memory being coupled to the processors; the memory storing one or more computer program codes, the computer program codes including computer instructions; when the processor executes the computer instructions, the flattening angle detection device for the foldable device performs the following steps:

[0030] Obtain point cloud images of the first and second middle frames in a flattened state. The point cloud images include a first middle frame region corresponding to the first middle frame and a second middle frame region corresponding to the second middle frame. After converting the three-dimensional coordinates of the point cloud in the first middle frame region into two-dimensional coordinates, determine the two-dimensional coordinates of a pixel row based on the two-dimensional coordinates of the pixel rows and columns in the first middle frame region and perform line fitting to obtain a first fitted line. After converting the three-dimensional coordinates of the point cloud in the second middle frame region into two-dimensional coordinates, determine the two-dimensional coordinates of a pixel row based on the two-dimensional coordinates of the pixel rows and columns in the second middle frame region and perform line fitting to obtain a second fitted line. Calculate the first tilt angle between the first fitted line and the reference plane and calculate the second tilt angle between the second fitted line and the reference plane. Determine the flattened angle of the first and second middle frames based on the first and second tilt angles.

[0031] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps: receiving a point cloud map sent by a spectral imager, the point cloud map being obtained by the spectral imager based on the depth map and two-dimensional spatial information of the measured first and second middle frames in the flattened state.

[0032] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps: reducing or enlarging the point cloud map.

[0033] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0034] The scanning speed during point cloud imaging is obtained, the calibration scale of the preset calibration plate is set, and the z-coordinate transformation threshold is set; the z-coordinate transformation threshold is given by the spectral imager. The point cloud image is scaled down or enlarged according to the scanning speed, calibration scale, and z-coordinate transformation threshold. The point cloud image includes multiple point cloud three-dimensional coordinates. The product of the x-coordinate of the point cloud three-dimensional coordinates and the scanning speed is the scaled-down or enlarged x-coordinate. The product of the y-coordinate of the point cloud three-dimensional coordinates and the calibration scale is the scaled-down or enlarged y-coordinate. The product of the z-coordinate of the point cloud three-dimensional coordinates and the z-coordinate transformation threshold is the scaled-down or enlarged z-coordinate. The point cloud three-dimensional coordinates include the first point cloud three-dimensional coordinates and the second point cloud three-dimensional coordinates.

[0035] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0036] From each pixel row in the first frame region, arbitrarily select a pixel row and fit a straight line to the corresponding two-dimensional coordinate row of the point cloud to obtain a first fitted straight line; and from each pixel row in the second frame region, arbitrarily select a pixel row and fit a straight line to the corresponding two-dimensional coordinate row of the point cloud to obtain a second fitted straight line.

[0037] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0038] After converting the 3D coordinates of the point cloud within the first frame region into 2D coordinates, the first mean coordinates corresponding to each pixel column within the first frame region are determined. The x-coordinate in the first mean coordinates is the x-coordinate of each point cloud 2D coordinate within the corresponding pixel column, and the y-coordinate is obtained by summing and averaging the y-coordinates of each point cloud 2D coordinate within the corresponding pixel column. A first fitted line is obtained by fitting the first mean coordinates corresponding to each pixel column within the first frame region. The 2D coordinates of a determined pixel row are the first mean coordinates corresponding to each pixel column within the first frame region.

[0039] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0040] After converting the 3D coordinates of the point cloud within the second frame region into 2D coordinates, the second mean coordinates corresponding to each pixel column within the second frame region are determined. The x-coordinate in the second mean coordinates is the x-coordinate of each point cloud 2D coordinate within the corresponding pixel column, and the y-coordinate is obtained by summing and averaging the y-coordinates of each point cloud 2D coordinate within the corresponding pixel column. A second fitted line is obtained by fitting the second mean coordinates corresponding to each pixel column within the second frame region. The 2D coordinates of a determined pixel row are the second mean coordinates corresponding to each pixel column within the second frame region.

[0041] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0042] At least one region of interest (ROI) is selected within the first frame region. The 3D coordinates of the first point cloud within each ROI are converted into 2D coordinates. The x-coordinate in the 2D coordinates is the same as the x-coordinate in the 3D coordinates, and the y-coordinate is the same as the z-coordinate. For the 2D coordinates, the y-coordinates are summed column-wise according to the corresponding pixel columns in the ROI, and the average value is taken while keeping the x-coordinate unchanged, resulting in n first mean coordinates. Here, n is the number of pixel columns in the ROI, and n is a positive integer. The first mean coordinates corresponding to at least one ROI are then fitted with a straight line using the least squares method to obtain the first fitted straight line corresponding to at least one ROI.

[0043] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0044] At least one second region of interest (ROI) is selected within the second frame region. The 3D coordinates of the second point cloud within each ROI are converted into 2D coordinates. The x-coordinate in the 2D coordinates is the same as the x-coordinate in the 3D coordinates, and the y-coordinate is the same as the z-coordinate. For the 2D coordinates, the y-coordinate is summed column-wise according to the pixel column of the corresponding ROI, and the average value is taken while keeping the x-coordinate unchanged, resulting in m second mean coordinates. Here, m is the number of pixel columns in the ROI, and m is a positive integer. The second mean coordinates corresponding to at least one ROI are then fitted with a line using the least squares method to obtain the second fitted line corresponding to at least one ROI.

[0045] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps: randomly selecting at least one first region of interest from the first middle frame region according to a preset first region range, wherein the size of the first region of interest corresponds to the preset region size.

[0046] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps: randomly selecting at least one second region of interest from the second middle frame region according to a preset second region range, wherein the size of the second region of interest corresponds to the preset region size.

[0047] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0048] Determine the starting and ending coordinates of the first fitted line, and use the inverse trigonometric function atan2 to calculate the first tilt angle between the first fitted line and the reference plane. Determine the starting and ending coordinates of the second fitted line, and use the inverse trigonometric function atan2 to calculate the second tilt angle between the second fitted line and the reference plane. Calculate the difference between the first tilt angle corresponding to the first fitted line and the second tilt angle corresponding to the second fitted line to obtain the flattening angle of the first and second middle frames in the flattened state.

[0049] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following step: outputting flattening angle indication information.

[0050] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps: outputting flattening angle size information; and / or, when the flattening angle is not within a preset angle range, determining that the flattening state of the first and second middle frames is unqualified, and outputting a prompt message.

[0051] In one possible implementation of the second aspect, when the aforementioned computer instructions are executed by the processor, the flattening angle detection device of the foldable device further performs the following steps:

[0052] After converting the 3D coordinates of the points in the first and second middle frame regions of the point cloud map into 2D coordinates, smoothing filtering is applied to the first and second middle frame regions of the point cloud map.

[0053] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor in an electronic device, causes a foldable device flattening angle detection device to perform a foldable device flattening angle detection method as described in the first aspect and any possible implementation thereof.

[0054] Fourthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method described in the first aspect and any possible implementation thereof. The computer may be the flattening angle detection device for the aforementioned foldable device.

[0055] Understandably, the beneficial effects that the flattening angle detection device of the foldable device of any possible implementation of the second aspect, the computer-readable storage medium of the third aspect, and the computer program product of the fourth aspect can achieve can be referred to as the beneficial effects of the first aspect and any possible implementation thereof, which will not be repeated here. Attached Figure Description

[0056] Figure 1 This application provides a product form for a foldable device. Figure 1 ;

[0057] Figure 2 This application provides a product form for a foldable device. Figure 2 ;

[0058] Figure 3 This application provides a product form for a foldable device. Figure 3 ;

[0059] Figure 4 This is a schematic diagram of the structure of a foldable device provided in an embodiment of this application;

[0060] Figure 5 This is a schematic diagram of the structure of a detection device provided in an embodiment of this application;

[0061] Figure 6 A system architecture diagram of a flattening angle detection system for a foldable device provided in this application embodiment;

[0062] Figure 7 A schematic diagram of a depth map provided in an embodiment of this application;

[0063] Figure 8 A flowchart illustrating a method for detecting the flattening angle of a foldable device provided in an embodiment of this application;

[0064] Figure 9 A schematic diagram of a point cloud map provided in an embodiment of this application;

[0065] Figure 10 A schematic diagram of a spectral imaging ratio provided in an embodiment of this application;

[0066] Figure 11 A schematic diagram of a point cloud image captured in an embodiment of this application;

[0067] Figure 12 A schematic diagram illustrating the division of a mid-frame region as provided in an embodiment of this application;

[0068] Figure 13 A schematic diagram illustrating ROI selection as provided in an embodiment of this application;

[0069] Figure 14 A schematic diagram of a linear fitting process provided in an embodiment of this application;

[0070] Figure 15 A point cloud diagram of a straight line fitting provided in an embodiment of this application;

[0071] Figure 16 A schematic diagram of a curve for straight line fitting provided in an embodiment of this application;

[0072] Figure 17 A schematic diagram of a flattened angle provided for an embodiment of this application;

[0073] Figure 18 This is a schematic diagram illustrating the comparison of curves before and after smoothing filtering, provided in an embodiment of this application. Detailed Implementation

[0074] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is only for the purpose of describing specific embodiments and is not intended to limit the application. Furthermore, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., are not necessarily different. Also, in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0075] With the development of electronic device technology, various foldable devices have emerged, such as foldable phones. For example, Figures 1-3 The diagrams show product forms of foldable phones. Figure 1 The diagram shows a product form of a foldable phone that folds inward horizontally (hereinafter referred to as an inward foldable phone). Figure 2 The image shows a product form diagram of a foldable phone that folds outwards horizontally (hereinafter referred to as an outward-folding phone). Figure 3 The diagram shows a product form of a vertically folding inward folding phone (hereinafter referred to as a vertically folding phone).

[0076] like Figure 1 As shown, an inward-folding phone includes an outer screen and an inner screen. The inner screen is a foldable display that folds down to form screen A and screen B. The outer screen is located on the back of the inner screen; for example, it can be located behind screen A or screen B. Figure 1The external screen of the inward-folding phone shown is located on the back of screen B. (As shown...) Figure 1 As shown, the form factors of inward-folding phones include Figure 1 The unfolded state (unfolded state) shown in (1) is as follows: Figure 1 The semi-folded state shown in (2) (between the folded and fully folded states), and Figure 1 The fully folded state shown in (3).

[0077] like Figure 2 As shown, an outward-folding phone includes a foldable display screen, which can be folded to form screen A and screen B. Figure 2 As shown, the form factors of outward-folding phones include Figure 2 The unfolded state (unfolded state) shown in (1) is as follows: Figure 2 The semi-folded state shown in (2) (between the folded and fully folded states), and Figure 2 The fully folded state shown in (3).

[0078] like Figure 3 As shown, a vertically folding phone also includes an outer screen and an inner screen. Similarly, the inner screen is a foldable display, and when folded, it can form screen A and screen B. Furthermore, the outer screen of a vertically folding phone cannot be folded, and it can also be placed on the back of screen A or screen B. Figure 3 The outer screen shown is located on the back of screen A. (As shown) Figure 3 As shown, the forms of vertically folding phones include Figure 3 The unfolded state (unfolded state) shown in (1) is as follows: Figure 3 The semi-folded state shown in (2) (between the folded and fully folded states), and Figure 3 The fully folded state shown in (3).

[0079] The above Figures 1-3 The foldable devices shown are able to fold because they typically include a hinge assembly. For example, with... Figure 1 Taking the inward-folding phone shown as an example, Figure 4 A schematic diagram of a foldable device is shown.

[0080] like Figure 4 As shown, the foldable phone includes a mid-frame 401, a mid-frame 402, and a hinge assembly 403. The mid-frame can be understood as the frame located between the device screen and the back cover, supporting various internal components. For example, the device battery, motherboard, camera, ribbon cables, various sensors, and microphone are typically mounted on a frame or shell, and this frame or shell is the mid-frame. In other words, in this embodiment... Figure 4The middle frame 401 and middle frame 402 shown can be understood as the parts located between the phone screen and the phone back cover. Figure 4 (Not explicitly shown) The frame or housing in the middle used to install mobile phone components.

[0081] like Figure 4 As shown, the middle frame 401 is connected to one side of the hinge assembly 403, and the middle frame 402 is connected to the other side of the hinge assembly 403. Figure 4 As shown, the middle frame 401 is connected to the left side of the hinge assembly 403, and the middle frame 402 is connected to the right side of the hinge assembly 403. It should be noted that... Figure 4 The connection positions shown are merely examples for the embodiments of this application, and the embodiments of this application do not constitute any limitation on the connection positions or connection methods of any components in the foldable device. For example, the middle frame 401 may be connected to the right side of the hinge assembly 403, while the middle frame 402 may be connected to the left side of the hinge assembly 403.

[0082] After the middle frame 401, middle frame 402, and pivot assembly 403 are connected, the middle frame 401 and middle frame 402 can rotate based on the pivot assembly 403. Through rotation, the middle frame 401 and middle frame 402 can form different shapes. For example, they can form... Figure 1 The flattened state shown in (1), or the formation Figure 1 The semi-folded state shown in (2) or the formation Figure 1 The fully folded state shown in (3) is an example. In general, the reason why foldable phones can form flat and folded states (including half-folded and fully folded states) is mainly based on hinge components.

[0083] Furthermore, foldable devices (such as the foldable phone mentioned above) can further implement the required functions based on the folding angle. For example, foldable devices can further control the display state of the screen based on the angle formed by the fold, including switching between different displays or changing the displayed content based on the folding angle.

[0084] For example, with Figure 1 Taking the foldable phone shown as an example, when the foldable phone is in the... Figure 1 In the unfolded state shown in Figure (1), the foldable phone can drive the inner screen to display the application interface, while when the foldable phone is unfolded... Figure 1 The flattened state shown in (1) is switched to Figure 1 After the fully folded state shown in (3), the foldable phone can drive the switching of the outer screen to display the application interface.

[0085] However, since different models or even different foldable devices of the same model mostly execute the same set of business logic to achieve the required functions based on the folding angle, if the form of the foldable device (including unfolded and folded states) is not uniformly controlled within the same range of error, it may affect the accuracy of the business logic execution.

[0086] Therefore, to ensure that foldable devices can accurately achieve the required functions based on the folding angle, and to ensure that foldable devices have a reasonable and consistent aesthetic appearance in both flattened and folded states, manufacturers currently perform corresponding tests on the shape of the foldable device after assembling the mid-frame and rotating components and connecting all parts. This ensures that the shape of the assembled foldable device is correctly and uniformly within a reasonable range, thereby avoiding unreasonable shapes caused by assembly errors.

[0087] Traditionally, the detection of the angle formed by foldable devices in their flattened state (i.e., the flattened angle) is mainly achieved through spatial normal vectors. For example, using... Figure 4 Taking the middle frame shown as an example, traditionally, the spatial normal vectors of the middle frame 401 and the middle frame 402 can be calculated first, and then the angle formed by the corresponding spatial normal vectors of the middle frame 401 and the middle frame 402 can be calculated. The angle formed by the corresponding spatial normal vectors is used as the detected flattening angle. Then, it can be further determined whether the flattened state of the foldable device is within a reasonable range based on this flattening angle.

[0088] However, a plane can typically have an infinite number of normal vectors, so the normal vectors for middle frame 401 and middle frame 402 are not unique. Therefore, if the normal vectors are not chosen appropriately, it can easily affect the detection of flattened angles, thereby reducing the accuracy of flattened angle detection.

[0089] In summary, in order to improve the accuracy of flattening angle detection of foldable devices, to ensure the accuracy of subsequent function implementation of foldable devices, and to ensure that foldable devices can have a reasonable and uniform aesthetic appearance based on the flattening angle, this application provides a method for flattening angle detection of foldable devices. This method can be applied to a flattening angle detection device for foldable devices (hereinafter referred to as the detection device).

[0090] The flattening angle detection method for foldable devices provided in this application embodiment is mainly achieved by the detection device acquiring a point cloud map of the first and second middle frames of the foldable device in a flattened state through the pivot assembly.

[0091] Specifically, since the point cloud map can accurately describe the three-dimensional structure of a scene or an object, in the embodiments of the present application, the three-dimensional structure of the first middle frame and the second middle frame of the foldable device in the flattened state represented by the point cloud map is used to directly detect the flattened angle between the first middle frame and the second middle frame in the foldable device (such as Figure 4 the middle frame 401 and the middle frame 402 shown).

[0092] In this way, compared with the traditional method of indirectly detecting the flattened angle by selecting the spatial normal vectors of the two middle frames and regarding the angle formed by the two spatial normal vectors as the flattened angle, in the embodiments of the present application, the flattened angle between the first middle frame and the second middle frame (that is, the angle between the first middle frame and the second middle frame in the flattened state) is directly calculated through the point cloud map, making the detection of the flattened angle more direct, and thus the detection of the flattened angle more accurate.

[0093] Furthermore, the detection device can also determine whether the flattened state of the first middle frame and the second middle frame after the foldable device is assembled is within a reasonable range according to the detected flattened angle, so as to further determine whether the flattened state of the first middle frame and the second middle frame in the foldable device is qualified. If the detection device detects that the flattened state of the first middle frame and the second middle frame of the assembled foldable device is not within a reasonable range, then the detection device can determine that this assembly is unqualified, and further the detection device can output a prompt message to prompt the user (such as the assembly user, the quality inspection user, etc.) that the product is unqualified, facilitating correct reassembly. In this way, it is possible to avoid the problem of unreasonable flattened state of the foldable device caused by incorrect assembly of the middle frame.

[0094] In addition, in the embodiments of the present application, the detection of the flattened state is realized based on the point cloud map. If the user has any doubts about the detection of the detection device in the future, the user can also export the image used for detection (such as directly exporting the point cloud map or exporting the processed point cloud map), so as to directly view the three-dimensional structure of the first middle frame and the second middle frame of the foldable device in the flattened state, prevent detection errors of the detection device, and facilitate the user to observe and trace the detection of the flattened angle.

[0095] In some embodiments, the above-mentioned detection device, that is, the flattened angle detection device of the foldable device, can be a server, a terminal, a computer device, etc. For example, the terminal can include at least one of a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a tablet computer, a mobile phone, and a foldable electronic device. It can be understood that the specific type of the detection device in the embodiments of the present application is not particularly limited.

[0096] Exemplarily, Figure 5 shows a schematic structural diagram of a detection device.

[0097] like Figure 5 As shown, the detection device may include at least one processor, a memory, and at least one interface circuit. The processor is connected to the memory. Furthermore, the processor and the interface circuit can be interconnected via wiring.

[0098] The aforementioned processor may include one or more processing units, such as a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0099] In this embodiment, the processor can generate operation control signals based on the instruction opcode and timing signals to control instruction fetching and execution. In some embodiments, the processor can act as an execution entity to implement the flattening angle detection method for a foldable device described in any embodiment of this application.

[0100] For example, the processor can acquire point cloud images of the first and second mid-frames in the foldable device in their flattened state. Then, the processor uses these point cloud images to detect the flattened angles of the first and second mid-frames, thereby improving the accuracy of the flattened angle detection. Simultaneously, the processor can further determine whether the flattened state formed by the first and second mid-frames in the foldable electronic device is acceptable based on the detected flattened angles, preventing issues caused by incorrect mid-frame assembly leading to an unreasonable flattened state.

[0101] In some embodiments, the memory may also be located within the processor for storing instructions and data. In some embodiments, the memory in the processor may be a cache memory. This memory can store instructions or data that the processor has used or that are used frequently. If the processor needs to use the instruction or data, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0102] Interface circuits can be used to receive signals or data from other devices, and also to send signals or data to other devices. For example, the testing equipment can receive point cloud images of the foldable device in its flattened state from other devices (such as spectral imagers, LiDAR, and 3D cameras) via the interface circuit, thus acquiring the point cloud image. As another example, if the testing equipment determines that the flattened state of the foldable device is unqualified, it can send a prompt message to other terminals (such as those used by assembly users or quality inspection users) via the interface circuit to indicate that the foldable device assembly is unqualified. Alternatively, the interface circuit can output the size of the flattened angle to other terminals to inform the user of the detected flattened angle.

[0103] In other embodiments, the detection device may also include any one or more of the following: an external storage interface, an internal memory, a universal serial bus (USB) connector, a charging management module, a power management module, a battery, a display screen, an antenna, a mobile communication module, a wireless communication module, an audio module, a speaker, a receiver, a microphone, a headphone jack, a sensor module, buttons, a motor, an indicator, a camera module, and a user identification module (SIM) card interface. This application does not impose any limitations on these aspects.

[0104] It is understood that the structure of the detection device illustrated in the embodiments of this application does not constitute a specific limitation on the detection device. In other embodiments of this application, the detection device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0105] Figure 6 A system architecture diagram of a flattening angle detection system for a foldable device is shown.

[0106] like Figure 6 As shown in the diagram, the system architecture includes a spectral imaging system, a detection device, and a foldable device for measurement and imaging. The spectral imaging system includes a spectral imager, a spectral confocal sensor, a lens group L, and a reflector.

[0107] The spectral imager is communicatively connected to the detection equipment. The foldable device is placed below the lens group L. It is understood that, since the embodiments of this application mainly focus on the assembly stage of the foldable device, detecting the flattened state of the first and second mid-frames of the foldable device, therefore... Figure 6 As shown, the foldable device for measurement imaging in this application embodiment may only include a first middle frame and a second middle frame connected by a rotating component and in a flattened state.

[0108] A spectral imaging system is a spectral imaging technology that can simultaneously detect the two-dimensional spatial information (x, y) and one-dimensional spectral information λ of a measured object. Its core principle is the integration of spectrum and image, meaning that while acquiring image data, it also obtains the spectral information of each pixel in the image. Compared to ordinary 3D cameras, spectral imaging systems primarily obtain the depth value of the measured object through spectral technology. Simply put, a spectral imaging system can be viewed as a 3D camera with spectral analysis capabilities.

[0109] Specifically, when a spectral imaging system performs measurement imaging, such as Figure 6 As shown, a beam of white light (polychromatic light) is first emitted by a spectral confocal sensor. The white light emitted by the spectral confocal sensor passes through a small aperture S1 and illuminates the lens group L (dispersive lens group). The lens group L decomposes the white light into monochromatic light of different wavelengths, such as λ0, λi, and λn. Each wavelength corresponds to a fixed distance value and is focused at a point, thus forming different focal points S2 (λi).

[0110] Therefore, when the object being measured is placed in the measurement area, monochromatic light of specific wavelengths can illuminate its surface and be reflected. For example... Figure 6 As shown, the first and second middle frames, in their flattened state, are placed below the lens group L. Monochromatic light of different wavelengths formed by the lens group L can then illuminate the surfaces of the first and second middle frames and be reflected by the mirrors. The reflected light from the mirrors can then be further sensed by the spectral imager through the pinhole S3. In the principle of spectral confocal imaging, only reflected light perfectly focused on the surface of the object being measured can pass through the pinhole S3. Figure 6 As shown, the dashed line formed by the reflection is the reflected light that did not pass through the small hole S3.

[0111] After the spectral imager senses the reflected light, it analyzes the spectrum to determine the wavelength λ of the reflected light passing through the pinhole S3. This wavelength λ allows it to calculate the distance between the surface of the object being measured and the lens group L. Therefore, the spectral imager can determine the depth (height) z(x, y) of each pixel on the surface of the object being measured (e.g., the first and second middle frames), thus obtaining a depth map of the object. For example, Figure 7 A schematic diagram of a depth map is shown. Figure 7 The image shown is a grayscale representation of the corresponding depth map. Different grayscale colors correspond to different wavelengths, representing different depth values ​​z(x, y). In other words, regions with different grayscale values ​​have different depth values ​​z(x, y).

[0112] Furthermore, while emitting white light, the spectral confocal sensor is driven by a motor to move at a certain speed. This motor movement directs the emitted white light through the lens group L onto every area of ​​the first and second middle frames, allowing for the measurement and imaging of all surface areas of the first and second middle frames. In other words, the motor movement scans the surfaces of the first and second middle frames point-by-point in the x and y directions to obtain the corresponding two-dimensional spatial information (x, y). Therefore, the spectral imager can perform spectral imaging based on the measured depth value z(x, y) and the two-dimensional spatial information (x, y), thus obtaining a point cloud map of the measured object. Since the point cloud map is a three-dimensional data matrix including the depth value z(x, y), the coordinates of a pixel (x, y) in the point cloud map can be denoted as (x, y, z(x, y)). (x, y) represents the two-dimensional spatial dimension. z(x, y) corresponds to the spectral information λ and is determined by λ; therefore, z(x, y) represents the one-dimensional spectral dimension, which is the pixel's depth value (height).

[0113] It should be noted that the image scanning method described above, which involves first acquiring the spectral information λ of a point on the object under test (such as the first and second middle frames), and then obtaining the imaging spectrum by scanning point by point in the x and y directions, is a point scan (whisk broom), or point-by-point scanning. In some embodiments, the image scanning method of the imaging spectral system may also include line scan (push broom) and area scan. Line scan is line-by-line scanning. During the measurement process, the spectral information (x, λ) of each point on a line on the object under test is acquired first, and then the spectral image (x, y, z(x, y)) is obtained by scanning line by line in the y direction. Area scan, on the other hand, acquires the spectral information of a cross-section. The specific image scanning method described above can be referred to in existing spectral imaging principles, and the embodiments of this application do not limit it in any way.

[0114] The following will describe in detail the method for detecting the flattening angle of a foldable device according to the embodiments of this application, with reference to the accompanying drawings. It should be noted that the flattening angle detection methods for foldable devices in the following embodiments can all be implemented in a detection device equipped with the aforementioned hardware structure.

[0115] Figure 8 A flowchart illustrating a method for detecting the flattening angle of a foldable device is shown, including steps S801-S804. Figure 8 The method for detecting the flattening angle of the foldable device shown can be applied to the aforementioned detection equipment. The following, in conjunction with... Figure 8 The method for detecting the flattening angle of a foldable device provided in the embodiments of this application will be described in detail.

[0116] S801, the detection device acquires point cloud images of the first and second middle frames in a flattened state.

[0117] It is understood that the first and second middle frames in the embodiments of this application can be understood as... Figure 4 The middle frame 401 and middle frame 402 are shown. For example, the first middle frame corresponds to middle frame 401 and the second middle frame corresponds to middle frame 402. Alternatively, the first middle frame corresponds to middle frame 402 and the second middle frame corresponds to middle frame 401.

[0118] In some embodiments, the first middle frame and the second middle frame can also be referred to as the main middle frame and the secondary middle frame, respectively. For example, the first middle frame (middle frame 401) is the main middle frame, and the second middle frame (middle frame 402) is the secondary middle frame. Alternatively, the first middle frame (middle frame 401) is the secondary middle frame, and the second middle frame (middle frame 402) is the main middle frame. It should be noted that the definitions of the main middle frame and the secondary middle frame can be set based on actual needs and usage habits, and this application embodiment does not impose any limitations on this. In a specific embodiment, the main middle frame and the secondary middle frame can be distinguished based on the installation position of important components.

[0119] For example, the mid-frame with a central processing unit (CPU) installed can be called the main mid-frame, and the mid-frame without a CPU installed can be called the secondary mid-frame. That is, if Figure 4 The CPU of the foldable phone shown is actually installed on the side of the mid-frame 401, and mid-frame 401 corresponds to the first mid-frame. Therefore, the first mid-frame can be called the main mid-frame, and the second mid-frame is naturally the secondary mid-frame. Alternatively, if... Figure 4 The CPU of the foldable phone shown is actually installed on the side of the middle frame 402, and the middle frame 402 corresponds to the second middle frame. Therefore, the second middle frame can be called the main middle frame, and the first middle frame is naturally the secondary middle frame.

[0120] For ease of distinction and description, in the following embodiments of this application, the image region in the point cloud corresponding to the first middle frame (such as middle frame 401) is referred to as the first middle frame region, and the image region in the point cloud corresponding to the second middle frame (such as middle frame 402) is referred to as the second middle frame region. That is, in the embodiments of this application, the point cloud includes a first middle frame region corresponding to the first middle frame and a second middle frame region corresponding to the second middle frame.

[0121] Point cloud images of the first and second middle frames in their flattened state (equivalent to the foldable device in its flattened state) can be acquired by other devices capable of point cloud acquisition. For example, these devices could include spectral imaging systems, LiDAR, 3D cameras, etc. After these other devices acquire the point cloud images of the first and second middle frames in their flattened state, they can send these images to a connected detection device. Thus, the detection device can obtain the point cloud images of the first and second middle frames in the flattened state of the foldable device.

[0122] In some embodiments, the detection device and the other devices described above can communicate with each other via wired or wireless means. For example, the detection device can connect to the spectral imaging system via Wi-Fi, Bluetooth, data cable, etc. It is understood that the connection method can be selected according to actual needs, and this application embodiment does not limit the connection method in any way.

[0123] In some embodiments, considering practical considerations, such as the fact that the frame of a foldable device may be made of metal, the acquisition of point clouds using LiDAR may be affected by factors such as metal glare and reflections. Therefore, embodiments of this application can be primarily based on the principle of spectral confocal imaging, utilizing, for example... Figure 6 The spectral imaging system shown is used to measure and image the point cloud maps of the first and second mid-frames in the foldable device in a flattened state, thereby ensuring the imaging effect of the point clouds of the first and second mid-frames in the foldable device as much as possible. The specific spectral imaging process of the point cloud map can be referred to the above embodiments. Figure 6 The description of the embodiments in this application will not be repeated here.

[0124] For example, Figure 9 A schematic diagram of a point cloud is shown.

[0125] Figure 9 The point cloud diagram shown is Figure 8 The depth map shown corresponds to the point cloud map. Understandably, this is to better illustrate point clouds at different altitudes. Figure 9 The point cloud diagram shown is a side view. It should be noted that... Figure 8 and Figure 9 The images shown are for illustrative purposes only and do not constitute a limitation on the image content. The specific image content of the depth map and the specific image content of the point cloud map depend on the structure of the actual foldable device being measured, and this application does not impose any limitations on them.

[0126] In summary, in order to accurately obtain the point cloud images of the first and second mid-frames in the flattened state, embodiments of this application can use a spectral imaging system to acquire the three-dimensional data of the first and second mid-frames in the flattened state and then perform imaging.

[0127] In some embodiments, although the imaging principles of spectral imagers are generally the same, the specific imaging processing logic may vary slightly between different manufacturers. For example, when transmitting images (such as point cloud maps), spectral imagers typically convert the height of the point cloud to an integer value according to a certain ratio before transmission. In this way, the height of the point cloud received by the detection device is the value adjusted proportionally by the spectral imager.

[0128] Meanwhile, due to the imaging ratio configured by the spectral imager itself, as well as the influence of factors such as the distance from the object being measured, the imaging ratio of the first and second middle frames presented in the point cloud map transmitted by the spectral imager may not be consistent with the actual ratio of the first and second middle frames in the foldable device.

[0129] For example, Figure 10 A schematic diagram of a spectral imaging scale is shown.

[0130] like Figure 10 As shown, Figure 10 The value shown in (1) is the actual size of the object being measured. Figure 10 The image in (2) shows the scale of the measured object after a long-distance spectral imaging process. Figure 10 As shown in (1) and (2), the first and second middle frames include some cut-out areas for mounting components. After long-distance spectral imaging, the scale of these cut-out areas is reduced compared to the actual scale. That is, if the imaging distance is relatively far, the scale of the object being measured will be reduced. Conversely, if the imaging distance is relatively close, the scale of the object being measured may be magnified.

[0131] Therefore, after the detection device acquires the point cloud image from the spectral imager, in order to ensure that the proportion presented by the point cloud image is consistent with the actual proportion of the first and second middle frames, and to ensure the accuracy of the flattening angle detection, the detection device in this embodiment of the application can further perform a scale conversion on the point cloud image (such as reducing or enlarging the point cloud image according to a certain proportion) to restore it to the actual proportion.

[0132] In one specific embodiment, the transformation expression for scaling the point cloud image by the detection device is as follows:

[0133] z2(x,y)=z1(x,y)*0.0001;

[0134] x2 = M * x1;

[0135] y2 = C * y1;

[0136] Where z1(x, y) is the z-coordinate (i.e., the height of the point cloud) before scaling, which is the height of the point cloud in the point cloud image transmitted by the spectral imager. z2(x, y) is the z-coordinate (i.e., the height of the point cloud) after scaling. 0.0001 is the z-coordinate transformation threshold, which needs to be determined according to the scaling factor used by the spectral imager to convert the height to an integer. 0.0001 is only used as an example in this application and does not constitute a limitation on the z-coordinate transformation threshold. In other words, the z-coordinate transformation threshold is mainly given by the spectral imager.

[0137] x1 and y1 are the x and y coordinates of the point cloud before scaling, and x2 and y2 are the x and y coordinates of the point cloud after scaling. M is the motor speed (i.e., the scanning speed / acquisition speed during point cloud imaging), and C is the calibration scale of the calibration plate. The motor speed M depends on the actual movement speed of the motor during imaging, and the calibration scale C is the actual image measurement and imaging process (e.g., ...). Figure 6 The calibration ratio of the calibration plate used in the measurement imaging scene shown.

[0138] For example, the motor speed M = 0.9 and the calibration ratio C = 0.5. However, it is understood that 0.9 and 0.5 are only used as examples in the embodiments of this application and do not constitute a limitation on the motor speed M and the calibration ratio C.

[0139] In some embodiments, in order to reduce the amount of data in subsequent image processing and improve the detection speed of the flattened state, after the detection device acquires the point cloud map, it can extract a portion of the point cloud map corresponding to a certain area from the complete point cloud map for subsequent processing.

[0140] Alternatively, to reduce the workload of the detection equipment, the point cloud map can be acquired only by capturing a portion of the area corresponding to the first and second middle frames, rather than the entire point cloud map. For example, when acquiring point cloud maps based on the principle of spectral confocal imaging, only a portion of the first and second middle frames can be exposed to the measurement area. In this way, the point cloud map acquired by the detection equipment from the spectral imager will only show this portion of the area exposed within the measurement area.

[0141] For example, Figure 11 A schematic diagram of a point cloud image is shown.

[0142] like Figure 11 As shown, to reduce the amount of data in subsequent image processing and improve the detection speed of the flattened state, the detection device can capture the point cloud image corresponding to this portion of the area within the dashed frame. Alternatively, it can only collect the point cloud images of the first and second middle frames corresponding to this portion of the area within the dashed frame. The area corresponding to middle frame 401 is the first middle frame area, and the area corresponding to middle frame 402 is the second middle frame area. This is understandable. Figure 11 The point cloud diagram shown is only used as an example in the embodiments of this application and does not constitute a limitation on the image content. The specific image content of the point cloud diagram depends on the structure of the actual foldable device being measured, and the embodiments of this application do not impose any limitations on it.

[0143] In other embodiments, if only a portion of the point cloud image is captured for flattening angle detection, the detection device can simultaneously capture point cloud images corresponding to multiple different regions, and then perform flattening state detection based on each point cloud image to obtain the flattening angle detection result for each point cloud image. If the flattening angle detection results corresponding to all captured point cloud images indicate that the flattened state of the foldable device is qualified, then the detection device can determine that the flattened state of the foldable device is qualified. Conversely, if any flattening state is unqualified among the flattening angle detection results corresponding to all point cloud images, then the detection device can determine that the flattened state of the foldable device is unqualified.

[0144] For example, Figure 12 A schematic diagram of a mid-frame region division is shown.

[0145] like Figure 12 As shown, the mid-frame of the foldable device (including the first and second mid-frames) can be divided into three regions, such as region ①, region ②, and region ③. Furthermore, if the point cloud image obtained by the detection device from the spectral imager completely includes these three regions, then when cropping the point cloud image, the detection device can crop one point cloud image from each of these three regions.

[0146] Alternatively, if the point cloud map obtained by the detection device from the spectral imager only includes any two of regions ①, ②, and ③, then the detection device can capture one point cloud map from each of these two regions when capturing the point cloud map.

[0147] For example, the testing device can acquire a point cloud map A corresponding to region ①, and then acquire a point cloud map B corresponding to region ③. The testing device then uses point cloud map A to check whether the flattened state formed by the first and second middle frames is acceptable. Simultaneously, the testing device also uses point cloud map B to check whether the flattened state formed by the first and second middle frames is acceptable. If the test results for both point cloud maps A and B are acceptable, then the testing device can determine that the flattened state of the foldable device is acceptable. Conversely, if the test results for both point cloud maps A and B show a case where the flattened state is unacceptable, then the flattened state of the foldable device is unacceptable.

[0148] This multi-region detection method for assessing the flattened angles of the first and second mid-frames in a foldable device improves accuracy compared to detecting only one region. For example, if one area of ​​the mid-frame is deformed or twisted, or if there is only one assembly error, that area may result in an unacceptable flattened state, while other areas may meet the requirements. Detecting only one area could potentially overlook this error. Therefore, simultaneously detecting flattened angles across multiple areas reduces the probability of errors and improves accuracy.

[0149] It is important to emphasize that regardless of whether the detection equipment captures a single point cloud image for flattened angle detection or captures multiple point cloud images for flattened state detection, since flattened angle detection mainly determines the flattened angle of the first and second middle frames, the point cloud image should include the first middle frame region corresponding to the first middle frame and the second middle frame region corresponding to the second middle frame.

[0150] S802, the detection device selects the first ROI and the second ROI in the point cloud map.

[0151] After obtaining the point cloud map of the foldable device in its flattened state, in order to reduce the amount of data and speed up the detection process, the detection device can select a region of interest (ROI) from the first middle frame region corresponding to the first middle frame and the second middle frame region corresponding to the second middle frame, thereby obtaining the first ROI corresponding to the first middle frame region and the second ROI corresponding to the second middle frame region.

[0152] In some embodiments, to ensure the accuracy of flattening angle detection, the detection device can determine a selection range within the middle frame region based on a given ROI range (i.e., a preset area range), and then randomly select an ROI within this determined selection range. Additionally, the detection device can also select an ROI of a corresponding size based on a given ROI size (i.e., a preset area size).

[0153] The given ROI range and ROI size can be pre-configured, and the specific values ​​of the ROI range and ROI size can be set according to actual business needs. This application embodiment does not impose any limitations on this. It is understood that since the selected range is determined based on the given ROI range, the actual selected range can be equal to the given ROI range.

[0154] For example, in a point cloud map, as follows Figure 11When showing the point cloud map of the cropped mid-frame region, since this point cloud map actually only includes a portion of the mid-frame region, giving an even smaller ROI range would further limit the selection range of the detection device, thus restricting the selection of ROI. In this case, the given ROI range, i.e., the selection range, can be directly the first and second mid-frame regions in this point cloud map. That is to say, in this case, the preset region range is equal to the first and second mid-frame regions.

[0155] For example, with Figure 11 Taking the point cloud diagram shown as an example, Figure 13 This diagram illustrates one method of ROI selection.

[0156] like Figure 13 As shown, the first ROI selected in this application embodiment can be... Figure 13 The ROIs shown are ROI 1, ROI 2, ROI 3, or ROI 4. The second ROI selected in this embodiment can be... Figure 13 The ROIs shown are ROI5, ROI6, ROI7, or ROI8. Among them, ROI1, ROI2, ROI3, and ROI4 are all within the first middle frame region (i.e., the given first ROI range, which is also the preset first region range), while ROI5, ROI6, ROI7, and ROI8 are also within the second middle frame region (i.e., the given second ROI range, which is also the preset first region range).

[0157] S803, the detection equipment fits the first fitting line corresponding to the first ROI and the second fitting line corresponding to the second ROI.

[0158] Whether a foldable electronic device is qualified in its flattened state is usually determined by detecting whether the flattened angle formed in the flattened state meets a preset angle. Therefore, the testing equipment can determine whether the flattened state of the foldable electronic device is qualified by detecting the flattened angle formed by the first and second middle frames in the flattened state.

[0159] Therefore, in order to determine the flattening angle of the first and second middle frames through the point cloud map, the detection device can convert the surface into a straight line and then calculate the tilt angle between the straight line and the reference plane.

[0160] That is, since the first ROI and the second ROI are partial image regions of the first and second middle frame regions, respectively, the first ROI and the second ROI are equivalent to partial cross-sections of the first and second middle frame surfaces. Therefore, after obtaining the first ROI and the second ROI, the detection device can first perform linear fitting on these two cross-sections to obtain the fitted straight lines corresponding to the cross-sections (the fitted straight lines can also be simply referred to as the cross-section lines), which means obtaining the first fitted straight line corresponding to the first ROI and the second fitted straight line corresponding to the second ROI.

[0161] Then, the testing equipment can accurately detect the angle between the first and second fitted lines and the reference plane (i.e., the flattening angle) in the flattened state based on the tilt angle between the first fitted line and the second fitted line. Thus, it can further determine whether the flattened state is qualified based on the flattening angle, thereby determining whether the flattened state of the foldable device including the first and second fitted lines is qualified.

[0162] In some embodiments, line fitting can be achieved by extracting point clouds within the ROI. That is, the first fitted line can be obtained by extracting point clouds within the first ROI and performing line fitting, while the second fitted line can be obtained by extracting point clouds within the second ROI and performing line fitting. Figure 14 A schematic diagram of a line fitting process is shown, including steps S1401-S1403.

[0163] The following, combined with Figure 14 The process of linear fitting in the embodiments of this application will be described in detail.

[0164] First, the detection equipment executes S1401 to convert the three-dimensional coordinates of the point cloud in the ROI into two-dimensional coordinates.

[0165] In this embodiment, the x-coordinate in the converted two-dimensional point cloud coordinates is the same as the x-coordinate in the corresponding three-dimensional point cloud coordinates. Simultaneously, the y-coordinate in the converted two-dimensional point cloud coordinates is the same as the z-coordinate in the corresponding three-dimensional point cloud coordinates. In other words, the process of converting three-dimensional point cloud coordinates to two-dimensional point cloud coordinates mainly involves: keeping the x-coordinate unchanged, and then using the z-coordinate from the three-dimensional coordinates as the y-coordinate in the two-dimensional coordinates. For example, assuming the three-dimensional point cloud coordinates are (x1, y1, z), then the converted two-dimensional point cloud coordinates are (x2, y2), where x2 = x1 and y2 = z.

[0166] Then, the detection device executes S1402, summing the y-coordinates of the point cloud 2D coordinates column by column (i.e., according to the pixel columns in the ROI) and taking the average value while keeping the x-coordinate unchanged, to obtain n mean coordinates. Here, n is the number of pixel columns in the ROI.

[0167] That is, for the two-dimensional coordinates of the point cloud in the ROI, the x-coordinate remains unchanged for each pixel column, but the detection device sums the y-coordinates column by column and takes the average value to obtain the mean coordinate of that column. If the ROI includes n pixel columns, then the mean coordinate of this ROI will include n values. Generally, n is a positive integer greater than 1.

[0168] For example, suppose the ROI contains three columns of pixels. The two-dimensional coordinates of the point cloud corresponding to the pixels in the first column are (x1, y1), (x1, y2), and (x1, y3); the two-dimensional coordinates of the point cloud corresponding to the pixels in the second column are (x2, y1), (x2, y2), and (x2, y3); and the two-dimensional coordinates of the point cloud corresponding to the pixels in the third column are (x3, y1), (x3, y2), and (x3, y3). Then, the detection device sums and averages the y-coordinates column by column while keeping the x-coordinate unchanged, thus obtaining three mean coordinates: (x1, (y1+y2+y3) / 3), (x2, (y1+y2+y3) / 3), and (x3, (y1+y2+y3) / 3).

[0169] For example, Figure 15 A schematic diagram of a point cloud fitted with a straight line is shown. (Example) Figure 15 As shown, Figure 15 Figure (1) shows the two-dimensional coordinates of the point cloud included within the ROI. Then, by... Figure 15 The two-dimensional coordinates of the point cloud shown in (1) are obtained by summing the y-coordinates column by column and taking the average value. Figure 15 The mean coordinates are shown in (2).

[0170] Finally, the detection equipment executes S1403, using the least squares method to perform linear fitting on these n mean coordinates to obtain the line corresponding to the ROI.

[0171] Least squares is a commonly used optimization method in statistics and mathematics, primarily used to fit data points to find the optimal functional relationship. This application embodiment uses least squares for line fitting, mainly relying on it to find the functional relationship between these n mean coordinates. It is understood that, besides least squares, other existing line fitting methods can also be used, such as the population mean method, multinomial fitting, and support vector machines, etc., and this application embodiment does not impose any limitations on these methods.

[0172] For example, Figure 16 A schematic diagram of a curve fitted to a straight line is shown.

[0173] like Figure 16 As shown, Figure 16 The dashed lines shown in (1) and (2) are the curves corresponding to the n mean coordinates. Figure 16The solid lines shown in (1) and (2) are the fitted lines obtained by fitting the dashed lines with straight lines.

[0174] In a specific embodiment, the fitting process for the first fitted line corresponding to the first ROI is as follows: First, the detection device converts the three-dimensional coordinates of the first point cloud within the first ROI into corresponding two-dimensional coordinates. The x-coordinate in the two-dimensional coordinates is the same as the x-coordinate in the three-dimensional coordinates, and the y-coordinate in the two-dimensional coordinates is the same as the z-coordinate in the three-dimensional coordinates. Then, the detection device sums the y-coordinates of the two-dimensional coordinates of the first point cloud column according to the pixel columns in the first ROI and takes the average value to obtain n first mean coordinates. Finally, the detection device uses the least squares method to perform linear fitting on these n first mean coordinates to obtain the first fitted line corresponding to the first ROI.

[0175] For example, in the embodiments of this application, the curves corresponding to these n first mean coordinates and the first fitted straight line can be referenced. Figure 16 As shown in (1) and (2). For example, the curves corresponding to these n first mean coordinates can be Figure 16 The dashed line in (1) can be the first fitted line. Figure 16 The solid line in (1). It is understood that the implementation of linear fitting for the first ROI in this embodiment can be specifically referred to the above. Figure 14 The description of the corresponding embodiments will not be repeated here.

[0176] In a specific embodiment, the fitting process for the second fitted line corresponding to the second ROI is as follows: First, the detection device converts the three-dimensional coordinates of the second point cloud within the second ROI into corresponding two-dimensional coordinates. The x-coordinate in the two-dimensional coordinates is the same as the x-coordinate in the three-dimensional coordinates, and the y-coordinate in the two-dimensional coordinates is the same as the z-coordinate in the three-dimensional coordinates. Then, the detection device sums the y-coordinates of the two-dimensional coordinates of the second point cloud column by column according to the pixel columns in the second ROI and takes the average value to obtain m second mean coordinates. Finally, the detection device uses the least squares method to perform linear fitting on these m second mean coordinates to obtain the second fitted line corresponding to the second ROI.

[0177] For example, in the embodiments of this application, the curves corresponding to the m second mean coordinates and the second fitted straight line can be referenced. Figure 16 As shown in (1) and (2). For example, the curves corresponding to these m second mean coordinates can be Figure 16 The dashed line in (2) and the second fitted line can be... Figure 16 The solid line in (2). It is understood that the implementation of linear fitting for the second ROI in this embodiment can be found in the above. Figure 14 The description of the corresponding embodiments will not be repeated here.

[0178] It should be noted that the value of n depends on the size of the first ROI, and the value of m depends on the size of the second ROI, which in turn depends on the number of pixel columns included in both the first and second ROIs. Understandably, when the sizes of the first and second ROIs are equal, n = m. Generally, the number of pixel columns is a positive integer, so n and m are usually positive integers greater than 1.

[0179] Understandably, if the impact of data volume on detection speed is not considered, the detection device can also obtain a first fitted line by fitting the coordinates of the entire first middle frame region, and a second fitted line by directly fitting the coordinates of the entire second middle frame region. In other words, a straight line can be fitted by summing and averaging the y-coordinates of all pixel columns in both the first and second middle frame regions. The specific process of this straight line fitting is the same as the process and principle of straight line fitting for the selected ROI described in the above embodiments, and will not be repeated here.

[0180] In other embodiments, when the detection device performs line fitting, after converting the three-dimensional coordinates of the point cloud in the ROI (or middle frame area) into two-dimensional coordinates of the point cloud, it is not necessary to sum the y-coordinates according to the pixel column to obtain a row of coordinates for line fitting. Instead, it can directly select any pixel row of point cloud two-dimensional coordinates in the ROI (or middle frame area) for line fitting to obtain the corresponding fitted line.

[0181] However, considering the unevenness of the surfaces of the first and second middle frames, which may lead to missing or abnormal pixels in the resulting point cloud image, the single-section line fitting method in this embodiment, which directly selects a pixel row and performs linear fitting on the two-dimensional coordinates of the selected pixel row, cannot avoid the influence of uneven surfaces compared to the multi-section line fitting method described in the above embodiment, which sums and averages the y-coordinates of the two-dimensional coordinates of the point cloud in each pixel column and then performs linear fitting.

[0182] Therefore, when the unevenness on the surfaces of the first and second middle frames is significant and likely to have an impact, the above-mentioned multi-section line fitting method can be used. However, when the unevenness on the surfaces of the first and second middle frames does not have an additional impact, in addition to the above-mentioned multi-section line fitting method, the single-section line fitting method of this application embodiment can also be used.

[0183] S804, the detection device calculates the tilt angle between the first fitted line and the second fitted line and the reference plane, and determines the flattening angle of the first middle frame and the second middle frame in the foldable device based on the tilt angle.

[0184] After the detection equipment obtains the first fitted line (i.e., the cross-sectional line corresponding to the first ROI) and the second fitted line (i.e., the cross-sectional line corresponding to the second ROI) through linear fitting, it can calculate the first tilt angle between the first fitted line and the reference plane, and the second tilt angle between the second fitted line and the reference plane. Then, based on the first and second tilt angles corresponding to the first and second fitted lines, the flattening angle formed by these two lines can be obtained, thereby determining the flattening angle of the first and second middle frames.

[0185] In some embodiments, the tilt angle between the fitted line and the reference plane can be any reference plane. However, for the accuracy of the flattening angle, it is necessary to ensure that the reference plane used to calculate the first fitted line and the second fitted line is the same plane. In a specific embodiment, since the point cloud map of the foldable device in the flattened state is usually obtained by measuring and imaging in contact with a horizontal plane, the tilt angle between the first fitted line and the second fitted line can specifically be the tilt angle with respect to the horizontal plane.

[0186] In some embodiments, the tilt angle between the fitted line and the horizontal plane can be determined using the inverse trigonometric function atan2. Specifically, the detection device can first determine the starting coordinates (xq, yq) and ending coordinates (xz, yz) of the fitted lines (including the first and second fitted lines). Then, based on the starting coordinates (xq, yq) and ending coordinates (xz, yz) of the fitted lines, the detection device uses the inverse trigonometric function atan2 to calculate the tilt angle of the lines (including the first and second fitted lines). Finally, the detection device calculates the difference between the corresponding tilt angles of the two lines; this difference is the flattening angle of the two lines, which is also the flattening angle of the first and second inner frames in the depth map. The expression for the tilt angle angle can be found below:

[0187] angle=atan2(yq-yz,xq-xz).

[0188] In this embodiment, if angle1 represents the tilt angle corresponding to the first fitted straight line and angle2 represents the tilt angle corresponding to the second fitted straight line, then the flattening angle = angle1 - angle2. Therefore, this embodiment can accurately calculate the flattening angle of the first and second middle frames based on a point cloud map, by fitting a straight line to a cross section, and based on the tilt angle between the fitted straight line (i.e., the cross section line) and the horizontal plane.

[0189] Understandably, although mathematically a straight line is a line without endpoints, meaning it has no beginning or end, in this embodiment, the fitted straight line (including the first and second fitted straight lines) is obtained by fitting a straight line based on finite point cloud two-dimensional coordinates. Therefore, the fitted straight line in this embodiment actually has endpoints. Thus, in this embodiment, the detection device can determine the starting point coordinates (xq, yq) and ending point coordinates (xz, yz) of the fitted straight line.

[0190] like Figure 17 The diagram shown illustrates a flattened angle. The following, in conjunction with... Figure 17 The method for detecting the flattened angle is explained.

[0191] like Figure 17 As shown, a first Region of Interest (ROI) is selected in the first frame region 401, and a second ROI is selected in the second frame region 402. The detection device performs linear fitting on the first ROI and the second ROI respectively to obtain a first fitted line corresponding to the first ROI and a second fitted line corresponding to the second ROI. The specific process of linear fitting can be found in the relevant description of the above embodiment, and will not be repeated here. Then, as... Figure 17 As shown, since the first fitted line corresponding to the first ROI is parallel to the horizontal plane, the first tilt angle calculated using the inverse trigonometric function atan2 is equal to 180°. Similarly, the second tilt angle between the second fitted line and the horizontal plane calculated using the inverse trigonometric function atan2 is equal to 1°. Furthermore, Figure 17 The flattened angle corresponding to the first and second middle frames shown is equal to 180°-1°=179°.

[0192] In some embodiments, after obtaining the flattened angles of the first and second middle frames, the detection device compares the flattened angles with a preset angle to determine whether the flattened state is qualified. Furthermore, when the flattened angles of the first and second middle frames meet the preset angle, the detection device determines that the flattened state of the foldable device is qualified. If the flattened angles of the first and second middle frames do not meet the preset angle, the detection device determines that the flattened state of the foldable device is unqualified.

[0193] Generally, when foldable devices are in a flattened state, their foldable displays (such as...) Figure 1 and Figure 3 The inner screen shown, or Figure 2 The foldable device's shape and appearance are most reasonable when the angle formed by the displayed screen (as shown) is 180 degrees. Therefore, the preset angle can be set based on 180 degrees.

[0194] For example, to be more precise, the preset angle can be a fixed value, such as 180 degrees. That is, when the flattening angle equals 180 degrees, the flattened state is considered acceptable. Conversely, if the flattening angle is not equal to 180 degrees, the flattened state is considered unacceptable. Alternatively, the preset angle can be a range, such as 179.5 degrees to 180 degrees. Then, when the flattening angle is greater than or equal to 179.5 degrees and less than or equal to 180 degrees, the flattened state is acceptable. Otherwise, if the flattening angle is less than 179.5 degrees or greater than 180 degrees, the flattening platform is considered unacceptable.

[0195] Next, in order to promptly correct any foldable devices that fail to meet the flattening requirements, the testing equipment can output a prompt message when it determines that the flattening of the foldable device is unqualified, such as pushing a prompt message to the corresponding user device. The prompt message is mainly used to alert the user, such as informing the quality inspection user or assembly user that the flattening of the first and second mid-frames is abnormal. The specific prompt content can be set according to needs, and this embodiment does not impose any limitations on it. Alternatively, after detecting flattening, the testing equipment can also directly output the size of the flattening angle to inform the user of the size of the flattening angle of the two mid-frames currently being tested.

[0196] In some embodiments, due to the influence of image acquisition, there may be some abnormal noise points or missing pixels in the point cloud image, and the resulting interference may affect the accuracy of flattened angle detection. Therefore, in order to remove noise interference and improve the accuracy of flattened angle detection, additional smoothing filtering can be applied to the image.

[0197] In one specific embodiment, the detection device can perform smoothing filtering on the 2D point cloud image after converting it from 3D to 2D. That is, after converting the 3D coordinates of the points in the point cloud image (such as the first ROI and / or the second ROI, or the first middle frame region and / or the second middle frame region mentioned above) to 2D coordinates, the detection device can perform smoothing filtering on the point cloud image (such as the first ROI and / or the second ROI, or the first middle frame region and / or the second middle frame region mentioned above) after coordinate conversion.

[0198] For example, taking the curve formed by the two-dimensional coordinates of the point cloud (including the two-dimensional coordinates of the first and second point clouds) in the point cloud image as an example, Figure 18 A schematic diagram showing a comparison of curves before and after smoothing filtering is provided. Figure 18 As shown in (1) and (2), before smoothing filtering, the curve is relatively tortuous due to abnormal noise points or missing pixels. After smoothing filtering, the curve is much smoother.

[0199] Alternatively, the detection device can perform smoothing filtering on the point cloud image before converting from 3D to 2D. However, since the embodiments of this application mainly calculate the flattening angle based on the first and second fitted lines corresponding to the two cross sections of the first and second middle frames, performing smoothing filtering after conversion to 2D will better avoid noise interference.

[0200] In some embodiments, to consider the flattened state of different regions of the mid-frame, the inspection device may also select multiple Regions of Interest (ROIs). That is, the inspection device may select multiple Regions of Interest (ROIs) corresponding to preset ROI sizes within the mid-frame region based on a preset ROI range. For example, multiple first ROIs may be selected from the first mid-frame region, or / and multiple second ROIs may be selected from the second mid-frame region.

[0201] For example, Figure 13 Taking the shown ROI as an example, the testing device can select multiple ROIs from ROI 1, ROI 2, ROI 3, and ROI 4 as the first ROI. For example, the first ROI includes ROI 2 and ROI 3. Simultaneously, the testing device can also select multiple ROIs from ROI 5, ROI 6, ROI 7, and ROI 8 as the second ROI. For example, the second ROI includes ROI 5 and ROI 8.

[0202] When multiple first ROIs are selected, the detection device can obtain the n first mean coordinates corresponding to each first ROI, and then use the least squares method to perform linear fitting on the n first mean coordinates corresponding to these multiple first ROIs together, thereby obtaining the first fitted line corresponding to these multiple first ROIs.

[0203] For example, the first ROI includes ROI2 and ROI3. The detection device uses the least squares method to fit the n first mean coordinates corresponding to ROI2 and the n first mean coordinates corresponding to ROI3 (a total of 2n first mean coordinates) together to obtain the first fitted line corresponding to the first middle frame region.

[0204] Similarly, when multiple second ROIs are selected, the detection device can obtain m second mean coordinates for each second ROI, and then use the least squares method to perform linear fitting on all m second mean coordinates for these multiple second ROIs, thereby obtaining the second fitted line for these multiple second ROIs. For example, if the second ROIs include ROI5 and ROI8, the detection device uses the least squares method to perform linear fitting on the m second mean coordinates corresponding to ROI5 and the m first mean coordinates corresponding to ROI8 (a total of 2m second mean coordinates), thereby obtaining the second fitted line corresponding to the second middle frame region. In other words, even if the detection device selects multiple first ROIs (or second ROIs), it can still perform linear fitting on all of them using the least squares method, thus fitting multiple first ROIs (second ROIs) into a single straight line.

[0205] Understandably, since the detection equipment performs line fitting based on point cloud two-dimensional coordinates (such as the first and second point cloud two-dimensional coordinates), and these point cloud two-dimensional coordinates maintain the x-coordinate of the point cloud three-dimensional coordinates while converting the z-coordinate of the point cloud three-dimensional coordinates into the y-coordinate of the two-dimensional coordinates, even if the selected ROIs are distributed in different locations or are far apart, the coordinate transformation removes the y-coordinate from the three-dimensional coordinates. Therefore, even if the distance between the selected ROIs is large, the influence of distance will be eliminated after the coordinate transformation, and the tilt angle of the fitted line will not be affected by the distance.

[0206] In this way, without affecting the tilt angle, multiple ROIs can be combined to consider the situation of multiple regions, which improves the accuracy of flattening angle detection compared to selecting a single ROI.

[0207] Another embodiment of this application provides a flattening angle detection device for a foldable device, comprising: one or more processors and a memory. The memory is coupled to the processor; the memory stores one or more computer program codes, the computer program codes including computer instructions; when the processor executes the computer instructions, the flattening angle detection device for the foldable device implements the flattening angle detection method for the foldable device described in any of the above embodiments.

[0208] Another embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor in a foldable device flattening angle detection device, causes the foldable device flattening angle detection device to implement the foldable device flattening angle detection method described in any of the above embodiments.

[0209] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps described in the above method embodiments.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0211] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0212] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0214] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0215] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting the flattening angle of a foldable device, characterized in that, The foldable device includes a first middle frame, a second middle frame, and a hinge assembly; the first middle frame and the second middle frame are respectively connected to the hinge assembly, and the first middle frame and the second middle frame are flattened through the hinge assembly; the method includes: Obtain point cloud maps of the first middle frame and the second middle frame in the flattened state, wherein the point cloud maps include a first middle frame region corresponding to the first middle frame and a second middle frame region corresponding to the second middle frame; Based on the two-dimensional coordinates of the point cloud corresponding to the rows and columns of pixels in the first middle frame region, a first fitted line is obtained by determining the two-dimensional coordinates of the point cloud of a pixel row and fitting a straight line; and, based on the two-dimensional coordinates of the point cloud corresponding to the rows and columns of pixels in the second middle frame region, a second fitted line is obtained by determining the two-dimensional coordinates of the point cloud of a pixel row and fitting a straight line; wherein, the two-dimensional coordinates of the point cloud are obtained by converting the three-dimensional coordinates of the point cloud in the first middle frame region and the second middle frame region. Calculate the first tilt angle between the first fitted line and the reference plane, and calculate the second tilt angle between the second fitted line and the reference plane. Determine the flattening angle of the first middle frame and the second middle frame based on the first tilt angle and the second tilt angle.

2. The method according to claim 1, characterized in that, The step of obtaining the point cloud images of the first and second middle frames in the flattened state includes: The system receives a point cloud image sent by a spectral imager, which is obtained by the spectral imager based on the depth map and two-dimensional spatial information of the measured first and second midframes in the flattened state.

3. The method according to claim 1, characterized in that, The method further includes: The scanning speed during point cloud imaging is obtained, the calibration ratio of the preset calibration plate is set, and the z-coordinate transformation threshold is obtained; wherein, the z-coordinate transformation threshold is given by the spectral imager; The point cloud image is scaled down or enlarged based on the scanning speed, the calibration ratio, and the z-coordinate transformation threshold. The point cloud image includes multiple point cloud three-dimensional coordinates. The product of the x-coordinate of the point cloud three-dimensional coordinates and the scanning speed is the scaled-down or enlarged x-coordinate. The product of the y-coordinate of the point cloud three-dimensional coordinates and the calibration ratio is the scaled-down or enlarged y-coordinate. The product of the z-coordinate of the point cloud three-dimensional coordinates and the z-coordinate transformation threshold is the scaled-down or enlarged z-coordinate. The point cloud three-dimensional coordinates include a first point cloud three-dimensional coordinate and a second point cloud three-dimensional coordinate.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the two-dimensional coordinates of a pixel row's point cloud based on the two-dimensional coordinates of the row and column pixels within the first frame region, and then performing line fitting to obtain the first fitted line, includes: After converting the three-dimensional coordinates of the point cloud within the first middle frame area into two-dimensional coordinates, the first mean coordinates corresponding to each pixel column within the first middle frame area are determined; wherein, the x-coordinate in the first mean coordinates is the x-coordinate of each two-dimensional coordinate of the point cloud within the corresponding pixel column, and the y-coordinate in the first mean coordinates is obtained by summing the y-coordinates of each two-dimensional coordinate of the point cloud within the corresponding pixel column and taking the average. The first mean coordinates corresponding to each pixel column within the first middle frame region are fitted with a straight line to obtain the first fitted straight line; wherein, the two-dimensional coordinates of the point cloud of a determined pixel row are the first mean coordinates corresponding to each pixel column within the first middle frame region.

5. The method according to any one of claims 1-3, characterized in that, The step of determining the two-dimensional coordinates of the point cloud of a pixel row based on the two-dimensional coordinates of the row and column pixels within the second frame region, and then performing line fitting to obtain the second fitted line, includes: After converting the three-dimensional coordinates of the point cloud within the second frame area into two-dimensional coordinates, the second mean coordinates corresponding to each pixel column within the second frame area are determined; wherein, the x-coordinate in the second mean coordinates is the x-coordinate of each two-dimensional coordinate of the point cloud within the corresponding pixel column, and the y-coordinate in the second mean coordinates is obtained by summing the y-coordinates of each two-dimensional coordinate of the point cloud within the corresponding pixel column and taking the average. The second mean coordinates corresponding to each pixel column within the second middle frame region are fitted with a straight line to obtain the second fitted straight line; wherein, the two-dimensional coordinates of the point cloud of a determined pixel row are the second mean coordinates corresponding to each pixel column within the second middle frame region.

6. The method according to claim 4, characterized in that, After converting the three-dimensional coordinates of the point cloud within the first middle frame area into two-dimensional coordinates, the first mean coordinates corresponding to each pixel column within the first middle frame area are determined. The first mean coordinates corresponding to each pixel column within the first middle frame region are fitted with a straight line to obtain the first fitted straight line, including: At least one first region of interest is selected within the first middle frame area, and the three-dimensional coordinates of the first point cloud within each first region of interest are converted into two-dimensional coordinates of the first point cloud; wherein, the x-coordinate in the two-dimensional coordinates of the first point cloud is the same as the x-coordinate in the three-dimensional coordinates of the first point cloud, and the y-coordinate in the two-dimensional coordinates of the first point cloud is the same as the z-coordinate in the three-dimensional coordinates of the first point cloud. For the first point cloud two-dimensional coordinates, according to the pixel column in the corresponding first region of interest, the y-coordinate is summed and averaged while the x-coordinate remains unchanged to obtain n first average coordinates; where n is the number of pixel columns in the first region of interest, and n is a positive integer; The first mean coordinates corresponding to the at least one first region of interest are fitted with a straight line using the least squares method to obtain the first fitted straight line corresponding to the at least one first region of interest.

7. The method according to claim 5, characterized in that, After converting the three-dimensional coordinates of the point cloud within the second middle frame area into two-dimensional coordinates, the second mean coordinates corresponding to each pixel column within the second middle frame area are determined. The second mean coordinates corresponding to each pixel column within the second middle frame area are fitted with a straight line to obtain the second fitted straight line, including: At least one second region of interest is selected within the second middle frame area, and the three-dimensional coordinates of the second point cloud within each second region of interest are converted into two-dimensional coordinates of the second point cloud; wherein, the x-coordinate in the two-dimensional coordinates of the second point cloud is the same as the x-coordinate in the three-dimensional coordinates of the second point cloud, and the y-coordinate in the two-dimensional coordinates of the second point cloud is the same as the z-coordinate in the three-dimensional coordinates of the second point cloud. For the second point cloud two-dimensional coordinates, according to the pixel column in the corresponding second region of interest, the y-coordinate is summed and averaged while the x-coordinate remains unchanged to obtain m second average coordinates; where m is the number of pixel columns in the second region of interest, and m is a positive integer; The second mean coordinates corresponding to the at least one second region of interest are fitted with a straight line using the least squares method to obtain the second fitted straight line corresponding to the at least one second region of interest.

8. The method according to claim 6, characterized in that, Selecting at least one first region of interest within the first middle frame region includes: At least one first region of interest is randomly selected from the first middle frame region according to a preset first region range, wherein the size of the first region of interest corresponds to the preset region size.

9. The method according to claim 7, characterized in that, Selecting at least one second region of interest within the second middle frame region includes: At least one second region of interest is randomly selected from the second middle frame area according to the preset second region range, and the size of the second region of interest corresponds to the preset region size.

10. The method according to any one of claims 1-3, characterized in that, The step of determining the point cloud two-dimensional coordinates of a pixel row based on the point cloud two-dimensional coordinates corresponding to the rows and columns of pixels in the first middle frame area and performing line fitting to obtain the first fitted line includes: arbitrarily selecting a pixel row from each pixel row in the first middle frame area, and performing line fitting on the point cloud two-dimensional coordinates corresponding to the selected pixel row to obtain the first fitted line. The step of determining the two-dimensional coordinates of a pixel row based on the two-dimensional coordinates of the point cloud corresponding to the rows and columns of pixels in the second middle frame region and performing line fitting to obtain the second fitted line includes: arbitrarily selecting a pixel row from each pixel row in the second middle frame region, and performing line fitting on the two-dimensional coordinates of the point cloud corresponding to the selected pixel row to obtain the second fitted line.

11. The method according to any one of claims 1-3, characterized in that, The step of calculating the first tilt angle between the first fitted line and the reference plane and calculating the second tilt angle between the second fitted line and the reference plane, and determining the flattening angle of the first middle frame and the second middle frame based on the first tilt angle and the second tilt angle, includes: Determine the starting point and ending point coordinates of the first fitted line, and use the inverse trigonometric function atan2 to calculate the first tilt angle between the first fitted line and the reference plane based on the starting point and ending point coordinates of the first fitted line. Determine the starting and ending coordinates of the second fitted line, and use the inverse trigonometric function atan2 to calculate the second tilt angle between the second fitted line and the reference plane based on the starting and ending coordinates of the second fitted line. The difference between the first tilt angle corresponding to the first fitted line and the second tilt angle corresponding to the second fitted line is calculated to obtain the flattening angle of the first middle frame and the second middle frame in the flattened state.

12. The method according to any one of claims 1-3, characterized in that, The method further includes: Output the size information of the flattening angle; and / or, When the flattening angle is not within the preset angle range, it is determined that the flattening state of the first middle frame and the second middle frame is unqualified, and a prompt message is output.

13. The method according to any one of claims 1-3, characterized in that, The method further includes: After the three-dimensional coordinates of the points in the first and second middle frame regions of the point cloud map are converted into two-dimensional coordinates, smoothing filtering is performed on the first and second middle frame regions of the point cloud map.

14. A flattening angle detection device for a foldable device, characterized in that, include: One or more processors and a memory, the memory being coupled to the processor; the memory storing one or more computer program codes, the computer program codes including computer instructions; When the processor executes the computer instructions, it causes the foldable device flattening angle detection device to perform the foldable device flattening angle detection method as described in any one of claims 1-13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor of the foldable device flattening angle detection device, the foldable device flattening angle detection device performs the foldable device flattening angle detection method as described in any one of claims 1-13.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor in the foldable device's flattening angle detection device, the foldable device's flattening angle detection device performs the flattening angle detection method of the foldable device as described in any one of claims 1-13.

Images