Group screen method, device, system and terminal equipment

Abstract

The present disclosure provides a method, device, system and terminal equipment for assembling a screen, wherein the method comprises: obtaining initial positioning information of a middle frame which has not been subjected to a dispensing operation; obtaining alternative positioning information of the middle frame which has been subjected to the dispensing operation; determining target positioning information of the middle frame based on the initial positioning information and the alternative positioning information; and performing an assembling operation on the middle frame and the screen based on the target positioning information. The method can effectively improve the positioning accuracy of the middle frame during the assembling operation, so that the screen can be more accurately attached to the middle frame.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent manufacturing technology, and in particular to a screen assembly method, apparatus, system and terminal equipment. Background Technology

[0002] Currently, in the screen assembly stage of equipment manufacturing, the industry requires visual edge detection of the mid-frame of the device to be assembled, in order to locate the mid-frame and thus mount the screen onto it. However, due to the process sequence, adhesive needs to be applied to the edges of the mid-frame before screen assembly. After the adhesive is applied, the edges of the mid-frame are covered by adhesive, which affects the visual edge detection of the mid-frame during the screen assembly stage, making it impossible to accurately locate the mid-frame and resulting in low assembly precision. Summary of the Invention

[0003] In view of this, the present disclosure provides a screen assembly method, apparatus, system and terminal device, which can effectively improve the accuracy of mid-frame positioning during screen assembly, so that the screen fits more accurately onto the mid-frame.

[0004] According to a first aspect of the present disclosure, a screen grouping method is provided, the method comprising:

[0005] Obtain the initial positioning information of the middle frame before glue dispensing;

[0006] Obtain alternative positioning information for the middle frame that has undergone dispensing operation;

[0007] Based on the initial positioning information and the alternative positioning information, the target positioning information of the middle frame is determined;

[0008] Based on the target positioning information, a screen assembly operation is performed on the mid-frame and the screen.

[0009] In any embodiment of this disclosure, the initial positioning information includes at least one of the following:

[0010] First position information in a first coordinate system is used to indicate the edge of the middle frame that has not undergone adhesive dispensing.

[0011] Second position information in a first coordinate system for indicating feature points on the middle frame where no dispensing operation has been performed; and / or,

[0012] The alternative location information includes at least:

[0013] The third position information of the feature points on the middle frame that have undergone dispensing operation is used to indicate the third position information in the second coordinate system.

[0014] In any embodiment of this disclosure, obtaining the initial positioning information of the mid-frame before adhesive dispensing includes:

[0015] The initial positioning information is obtained by the dispensing station, and the initial positioning information is used to indicate the path for the dispensing station to perform the dispensing operation;

[0016] The method further includes:

[0017] The initial positioning information is sent from the dispensing station to the screen assembly station;

[0018] Determining the target positioning information of the mid-frame based on the initial positioning information and the alternative positioning information includes:

[0019] The target positioning information is determined by the screen assembly station based on the initial positioning information.

[0020] In any embodiment of this disclosure, when the middle frame is conveyed to the screen assembly station in an assembly line manner after the glue dispensing operation at the glue dispensing station,

[0021] The step of obtaining the alternative positioning information of the middle frame that has undergone dispensing includes:

[0022] The alternative positioning information is obtained by the screen assembly station in conjunction with the vision system;

[0023] Determining the target positioning information of the mid-frame based on the initial positioning information and the alternative positioning information includes:

[0024] The target positioning information is determined by the screen assembly station based on the initial positioning information and the alternative positioning information.

[0025] In any embodiment of this disclosure, the step of determining the target positioning information of the mid-frame by the screen assembly station based on the initial positioning information and the alternative positioning information includes:

[0026] The screen assembly station determines a transformation matrix based on the second position information in the initial positioning information and the third position information in the alternative positioning information; wherein, the second position information is used to represent the position information of feature points on the middle frame that have not undergone glue dispensing in the first coordinate system; the third position information is used to represent the position information of feature points on the middle frame that have undergone glue dispensing in the second coordinate system; the transformation matrix is ​​used to indicate the mapping relationship between the first coordinate system and the second coordinate system;

[0027] Based on the transformation matrix, the first position information in the first coordinate system of the initial positioning information is transformed to the second coordinate system to obtain the fourth position information of the middle frame; the first position information is the position information of the edge of the middle frame without dispensing operation in the first coordinate system corresponding to the dispensing station;

[0028] The fourth location information is determined as the target location information.

[0029] In any embodiment of this disclosure, the step of performing a screen assembly operation on the mid-frame and the screen based on the target positioning information includes:

[0030] Based on the target positioning information, the target compensation displacement that the screen needs to move is determined;

[0031] After controlling the screen to move to compensate for the target displacement, a screen assembly operation is performed on the middle frame and the screen.

[0032] In any embodiment of this disclosure, determining the compensation displacement based on the target positioning information includes:

[0033] Based on the difference between the target positioning information and the preset positioning information of the middle frame, the first compensation displacement is calculated;

[0034] The second compensation displacement is calculated based on the difference between the screen positioning information and the preset positioning information of the screen.

[0035] The target compensation displacement is determined based on the first compensation displacement and the second compensation displacement.

[0036] In conjunction with any embodiment of this disclosure, the method further includes:

[0037] Obtain the target identification information of the middle frame that has undergone dispensing operation;

[0038] The process of obtaining the initial positioning information of the mid-frame before adhesive dispensing includes:

[0039] Based on the pre-determined association between different mid-frame identification information and different initial positioning information, the initial positioning information corresponding to the target identification information is obtained.

[0040] According to a second aspect of the present disclosure, a screen assembly device is provided, the device comprising:

[0041] The initial positioning module is used to obtain the initial positioning information of the middle frame before the glue dispensing operation;

[0042] Alternate positioning module, used to obtain alternative positioning information of the middle frame that has undergone dispensing operation;

[0043] The target positioning module is used to determine the target positioning information of the middle frame based on the initial positioning information and the alternative positioning information;

[0044] The screen grouping operation module is used to perform screen grouping operation on the middle frame and the screen based on the target positioning information.

[0045] According to a third aspect of the present disclosure, a screen assembly system is provided, the system comprising:

[0046] The first execution module obtains the initial positioning information of the middle frame before the glue dispensing operation, and reports the initial positioning information to the wire control module.

[0047] The dispensing module performs dispensing operations on the middle frame.

[0048] The wire control module is used to receive the initial positioning information reported by the first execution module, and to send the initial positioning information to the second execution module;

[0049] The second execution module is used to acquire alternative positioning information of the middle frame that has undergone dispensing operation, and to determine the target positioning information of the middle frame based on the initial positioning information and the alternative positioning information sent by the wire control module.

[0050] The screen grouping module performs screen grouping operations on the mid-frame and the screen based on the target positioning information.

[0051] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the screen grouping methods described in the first aspect.

[0052] According to a fifth aspect of the present disclosure, a terminal device is provided, comprising:

[0053] processor;

[0054] Memory used to store processor-executable instructions;

[0055] The processor is configured to execute the executable instructions to implement the steps of any of the screen grouping methods described in the first aspect above.

[0056] The technical solutions provided in this disclosure may have the following beneficial effects:

[0057] By determining the target positioning information of the mid-frame on which the screen assembly depends by using the initial positioning information of the mid-frame located before glue dispensing and the alternative positioning information located before screen assembly, the edge-finding positioning of the mid-frame is eliminated during the screen assembly stage. This avoids the problem of incorrect mid-frame edge recognition after glue dispensing at the mid-frame edge, thus improving the screen assembly accuracy. Moreover, the vision system that collects alternative positioning information before screen assembly does not require high precision, reducing equipment costs.

[0058] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0060] Figure 1 This disclosure is a flowchart illustrating a screen grouping method according to an exemplary embodiment;

[0061] Figure 2 This disclosure is a flowchart illustrating a screen grouping method according to an exemplary embodiment;

[0062] Figure 3 This disclosure is a schematic diagram of a screen assembly system according to an exemplary embodiment;

[0063] Figure 4 This is a schematic diagram illustrating a multi-screen pipeline scenario according to an exemplary embodiment of the present disclosure;

[0064] Figure 5 This disclosure is a block diagram of a screen assembly device according to an exemplary embodiment;

[0065] Figure 6 This is a schematic diagram of the structure of a terminal device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0066] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0067] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0068] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0069] With the development of intelligent manufacturing, the production and assembly of electronic devices can now be automated by machines in smart factories. The screen assembly (referred to as screen assembly) of electronic devices such as mobile phones, tablets, computers, and monitors is performed by screen assembly equipment (referred to as screen assembly equipment). For example, in the scenario of screen assembly on the pre-assembly production line of a smart factory, the technical solution used for assembling mobile phone screens is as follows: after the mid-frame and TP (Touch Panel) reach their respective operating stations, the screen assembly equipment is triggered to take pictures. The machine vision of the screen assembly equipment is used to locate the edges of the mid-frame and TP, determining the position of the mid-frame and TP to be assembled. Then, the positions are compared with preset positions to obtain two compensation values. Based on these two compensation values, the robotic arm of the screen assembly equipment is controlled to perform the screen assembly, so as to attach the TP to the position of the mid-frame.

[0070] This screen assembly method relies entirely on the visual accuracy of the assembly equipment itself, placing high demands on the accuracy of machine vision algorithms and the recognition precision of visual sensors. A significant challenge in the frame edge-finding and positioning process of this solution is the difficulty in edge grasping. This is manifested in several ways: rough edges, wear, or uneven paint application on the frame can all lead to edge grasping errors; different colored frames pose a significant challenge to the compatibility of the visual algorithm solution; and, specifically, due to the process sequence, glue application is required at the gluing station before the screen assembly station. After glue application, the actual edges of the frame are covered by glue, which affects the visual edge-finding process, making accurate positioning of the frame impossible and resulting in low screen assembly accuracy.

[0071] Based on this, this disclosure proposes a screen assembly method. By using the initial positioning information of the mid-frame before glue dispensing and the alternative positioning information before screen assembly, the target positioning information of the mid-frame on which the final screen assembly depends is obtained. There is no need to perform edge-finding positioning of the mid-frame during the screen assembly stage, avoiding the influence of glue dispensing on the positioning of the mid-frame and improving the screen assembly accuracy. Moreover, since this solution calls the positioning data during glue dispensing for mounting, the vision system in the screen assembly stage does not need to be so precise, reducing equipment costs.

[0072] like Figure 1 As shown, Figure 1This is a flowchart illustrating at least one embodiment of a screen assembly method, which can be used to assemble production equipment and includes the following steps:

[0073] In step 102, the initial positioning information of the middle frame before the glue dispensing operation is obtained.

[0074] The mid-frame refers to the area connecting the front panel and back cover of an electronic device. During the manufacturing process, after the mid-frame is installed, adhesive is applied to its edges at the dispensing station. This dispensing involves applying glue to the edges of the mid-frame so that the screen can be fixed to it after mounting. After dispensing, the mid-frame moves to the screen assembly station for screen assembly. The dispensing station is the operation point for applying glue to the mid-frame, while the screen assembly station is the operation point for assembling the screen onto the mid-frame. The mid-frame can be transported sequentially to the dispensing and assembly stations via a production line, or the dispensing and assembly operations can be performed separately by switching the operating modes of the dispensing and assembly stations without changing the position of the mid-frame.

[0075] Initial positioning information indicates the position of the mid-frame before adhesive dispensing. This can be the position of any point on the mid-frame, the position of multiple points on the mid-frame, or the position of a line on the mid-frame. If the initial positioning information is the position of a point on the mid-frame, it can include the coordinates of the point and the angle between the point and the horizontal plane. If the initial positioning information is the position of a line on the mid-frame, it can include the coordinates of the line and the angle between the line and the horizontal plane, or the angle between the line and another line on the horizontal plane. The initial positioning information can also include the position information of the mid-frame edges.

[0076] This embodiment does not limit the method of obtaining the initial positioning information. For example, the initial positioning information of the mid-frame can be obtained by using a camera to photograph the mid-frame and then detecting the obtained image; alternatively, the initial positioning information of the mid-frame can be obtained by using a 3D camera to scan the mid-frame and then recognizing the scanned data.

[0077] It should be noted that the mid-frame needs to be positioned before the dispensing operation. Based on the position information of the mid-frame edges obtained from this positioning, dispensing is performed on the mid-frame edges. This step can be performed after the mid-frame arrives at the dispensing station. The initial positioning information can include the position information of the mid-frame edges or can be used to calculate the position information of the mid-frame edges to facilitate dispensing at the mid-frame edges by the dispensing station. Alternatively, this step can be performed at an earlier operation station before the dispensing station, in which case the position information of the mid-frame edges required for dispensing is obtained separately after the process reaches the dispensing station.

[0078] In step 104, alternative positioning information of the middle frame that has undergone dispensing is obtained.

[0079] The alternative positioning information is used to indicate the position of the mid-frame that has undergone adhesive dispensing at the screen assembly station. This information can be the position of any point on the mid-frame, the position of multiple points on the mid-frame, or the position of a line on the mid-frame. Because the edges of the mid-frame are covered with adhesive after the adhesive dispensing operation at the station, making them difficult to identify, the alternative positioning information obtained in this step may not include information about the mid-frame edges. Since this step does not require collecting information about the mid-frame edges, the requirements for the vision system at the screen assembly station are very low, reducing the vision-related costs of the screen assembly station. For example, simpler vision algorithms and less precise vision sensors can be used.

[0080] Before the screen assembly operation begins on the mid-frame after adhesive dispensing, alternative positioning information of the mid-frame that has undergone adhesive dispensing is obtained. For example, the mid-frame can be photographed with a camera, and the obtained image can be detected to obtain alternative positioning information of the mid-frame; alternatively, the mid-frame can be scanned with a 3D camera, and the scanned data can be identified to obtain alternative positioning information of the mid-frame.

[0081] It should be noted that the initial positioning information and alternative positioning information include the position information of the same feature point on the middle frame, and the same feature point is not located on the edge of the middle frame, so that the feature point is not affected by the glue application. For example, the feature point can be a specific point or other shapes.

[0082] In step 106, the target positioning information of the middle frame is determined based on the initial positioning information and the alternative positioning information.

[0083] Target positioning information indicates the position of the screen at the edge of the mid-frame in the screen assembly station, instructing the screen assembly equipment to accurately place the screen onto the mid-frame. Since both the initial positioning information of the mid-frame obtained before the dispensing operation and the alternative positioning information of the mid-frame obtained before the screen assembly operation contain the position information of the same feature point, and this feature point is unaffected by the dispensing operation, the current position of the mid-frame edge at the screen assembly station—that is, the target positioning information—can be calculated by associating the position information of the same feature point in the initial and alternative positioning information with the position information of the mid-frame edge in the initial positioning information.

[0084] For example, since the initial positioning information is collected before dispensing, while the alternative positioning information is collected at the assembly station, and the actual position of the frame changes during transport on the production line, the target positioning information can be obtained by changing the position information of the same feature point in both the initial and alternative positioning information, thereby altering the position information of the frame edge in the initial positioning information. Specifically, if the alternative positioning information indicates that a specific point on the frame has shifted a distance *r* in a certain direction relative to its position in the initial positioning information, then the position information of the frame edge in the initial positioning information is also shifted a distance *r* in that direction to obtain the target positioning information.

[0085] For example, since the actual position of the middle frame has changed, the origin of the coordinate system used by the initial positioning information and the alternative positioning information is different. The screen assembly station cannot directly use the initial positioning information containing the position information of the middle frame edge with high precision. Therefore, it can perform coordinate system transformation by comparing and mapping the position information of the same feature point in the initial positioning information and the alternative positioning information, and obtain the target positioning information after the transformation of the position information of the middle frame edge contained in the initial positioning information.

[0086] For example, if the actual position of the middle frame has not changed, the target positioning information can be determined by the initial positioning information containing high-precision position information of the middle frame edge used in the dispensing operation.

[0087] In step 108, based on the target positioning information, a screen assembly operation is performed on the mid-frame and the screen.

[0088] The target positioning information indicates the actual position of the edge of the middle frame in the screen assembly station. When performing screen assembly operations, the screen can be moved to the position indicated by the target positioning information to perform the screen assembly operation.

[0089] Figure 2 Another screen assembly method provided in at least one embodiment of this disclosure, in order to further overcome the problem that when the actual position of the middle frame moves due to the conveying of the middle frame on an assembly line, the origin of the coordinate system used by the dispensing station and the screen assembly station to obtain the initial positioning information and alternative positioning information of the middle frame are different, and the screen assembly station cannot directly utilize the initial positioning information containing high-precision position information of the middle frame edge, the method may include the following processing, wherein, with Figure 1 The same steps as those in the process will not be described in detail again.

[0090] In step 202, the initial positioning information of the middle frame before the glue dispensing operation is obtained.

[0091] The initial positioning information includes at least one of the following: first position information in a first coordinate system for indicating the edge of the middle frame before glue dispensing; and second position information in a first coordinate system for indicating feature points on the middle frame before glue dispensing.

[0092] In this step, the initial positioning information can be obtained by the dispensing station. This initial positioning information is used to indicate the path for the dispensing station to perform the dispensing operation. For example, when the frame enters the dispensing station, before the dispensing operation, the first camera of the dispensing station above the frame scans the frame to locate the edge of the frame and obtain the first position information of the edge of the frame in the first coordinate system. When the first position information includes the coordinate information of the point and the angle information, it can be represented as (X1, Y1, R1). For example, the first position information can be the position information of the midpoint of the diagonal line connecting the frame, where X1 and Y1 represent the coordinate values ​​of the x-axis and y-axis in the first coordinate system, respectively, and R1 represents the angle between the diagonal line connecting the point and the xoy plane. Subsequent coordinates refer to this example. Thus, the edge information of the mid-frame can be represented by the position information of a point combined with the length and width values ​​of the mid-frame itself. Feature points on the mid-frame are identified to obtain the second position information of these feature points in the first coordinate system. This second position information refers to feature points located inside the mid-frame but not at its edge. There can be one or more feature points. When the second position information includes the coordinate and angle information of the feature points, it can be represented as (X2, Y2, R2). The first coordinate system is the coordinate system established from the perspective of the first camera. It can be a Cartesian coordinate system, or in other examples, a spatial Cartesian coordinate system. The first camera can use a high-precision 3D camera to obtain high-precision initial positioning information containing the first position information of the mid-frame edge, and to locate the mid-frame edge for high-precision dispensing. The dispensing station can use the initial positioning information to determine the position of the edge of the mid-frame to be dispensed, thereby determining the path for the dispensing operation and controlling the robotic arm to apply adhesive to the mid-frame along its edge. The initial positioning information is then sent from the dispensing station to the screen assembly station, so that the screen assembly station can utilize the edge position information that was not affected by the glue before dispensing when assembling the screen.

[0093] In one example, this step specifically includes: obtaining the initial positioning information corresponding to the target identification information based on the pre-determined association between the identification information of different mid-frames and different initial positioning information.

[0094] When the dispensing station obtains initial positioning information by scanning the mid-frame, it can also acquire the target identification information of the mid-frame and establish a correlation between the target identification information and the initial positioning information. This information is then uploaded to the wired control module, which can be understood as a module used to store direct communication between the dispensing station and the panel assembly station. The target identification information is used to uniquely identify the mid-frame and can be information such as an identification code or identifier. For example, after the mid-frame enters the dispensing station, before or after the dispensing operation, a scanner is used to scan the mounting code on the mid-frame to obtain the mid-frame's identification code.

[0095] In step 204, after the middle frame is dispensed by the dispensing station and then conveyed to the screen assembly station in an assembly line manner, the screen assembly station, in conjunction with the vision system, obtains the alternative positioning information.

[0096] The alternative positioning information includes at least: third position information in a second coordinate system for indicating the feature points on the mid-frame that have undergone adhesive dispensing. The vision system is a system comprising a camera in a screen assembly station and an algorithm for identifying and locating feature points in an image.

[0097] For example, after the middle frame enters the screen assembly station, before the screen assembly operation, the second camera on the screen assembly station above the middle frame captures an image of the middle frame, obtaining the third position information of the feature points on the middle frame in the second coordinate system. When the third position information includes the coordinate information and angle information of the point, it can be represented as (X20, Y20, R20). The second coordinate system is the coordinate system established from the perspective of the second camera. It can be a planar rectangular coordinate system, or in other examples, a spatial rectangular coordinate system, etc. Since the second camera only needs to capture the position information of the feature points and does not need to perform high-precision edge finding and positioning, the equipment requirements for the second camera are lower than those for the first camera. The vision system used in the screen assembly station also has lower precision requirements than the vision system of the dispensing station, reducing equipment costs.

[0098] In step 206, the screen assembly station determines the conversion matrix based on the second location information in the initial positioning information and the third location information in the alternative positioning information.

[0099] The transformation matrix is ​​used to indicate the mapping relationship between the first coordinate system and the second coordinate system. This embodiment does not limit the type of the transformation matrix. For example, when the first and second coordinate systems are planar rectangular coordinate systems, the transformation matrix can be a transformation matrix for 2D linear transformations, or it can be a transformation matrix for translation and affine transformations. Alternatively, when the first and second coordinate systems are spatial rectangular coordinate systems, the transformation matrix can be a transformation matrix for 3D transformations.

[0100] In this step, the second and third position information representing the same feature point are used to perform matrix calculations to obtain the transformation matrix M. This transformation matrix M can map the position of the first coordinate system to the position of the second coordinate system.

[0101] Continuing the previous example, this step also includes: obtaining the target identification information of the mid-frame that has undergone adhesive dispensing. When the mid-frame arrives at the assembly station after adhesive dispensing, the mid-frame is identified to obtain its target identification information. Based on the pre-determined association between different mid-frame identification information and different initial positioning information, the initial positioning information corresponding to the target identification information is obtained. For example, the assembly station uploads the identified target identification information to the wired control module and obtains the initial positioning information corresponding to the mid-frame to determine the transformation matrix, thereby ensuring the accuracy of the initial positioning information used each time the screen is assembled.

[0102] In step 208, based on the transformation matrix, the first position information in the first coordinate system of the initial positioning information is transformed to the second coordinate system to obtain the fourth position information of the middle frame; the fourth position information is determined as the target positioning information.

[0103] By performing matrix multiplication using the transformation matrix M and the first position information, the first position information in the first coordinate system can be converted into the fourth position information (X10, Y10, R10) in the local second coordinate system. The fourth position information is the target positioning information from the perspective of the second camera of the screen assembly station.

[0104] It should be noted that the transformation matrix can be used to transform both the coordinate and angle information in the first position information simultaneously, or it can transform the coordinate and angle information separately. When the first and second coordinate systems are in a Cartesian coordinate system on the same plane, it is also possible to transform only the coordinate information in the first position information to obtain the coordinate information in the target positioning information. When determining the angle information in the target positioning information, the angle information in the third position information and the second position information is subtracted to obtain the transformed angle R20-R2. The angle R1 in the first position information and the transformed angle are added to obtain the angle information R10 = R20-R2+R1 in the fourth position information.

[0105] In step 210, based on the target positioning information, the target compensation displacement that the screen needs to move is determined.

[0106] When assembling screens, the assembly station needs to control a robotic arm to move the screen above the middle frame because the middle frame and the screen are usually not in the same vertical position. The assembly station uses the converted target positioning information and a preset standard position to calculate the compensation displacement. The preset standard position is the position where the middle frame and screen are in the same vertical position. For example, the preset standard position can be set to (0, 0, 0), and the target compensation displacement is a movement of X10 along the x-axis, a movement of Y10 along the y-axis, and a rotation angle R10.

[0107] In one example, determining the compensation displacement based on the target positioning information includes:

[0108] Based on the difference between the target positioning information and the preset positioning information of the mid-frame, a first compensation displacement is calculated; based on the difference between the screen positioning information and the preset positioning information of the screen, a second compensation displacement is calculated; and based on the first compensation displacement and the second compensation displacement, the target compensation displacement is determined.

[0109] In this example, not only the middle frame needs to be positioned, but the screen controlled by the robotic arm also needs to be positioned. The difference between the target positioning information of the middle frame and the preset positioning information of the middle frame is the first displacement compensation value ▽1. The difference between the screen positioning information of the screen and the preset positioning information of the screen is the second displacement compensation value ▽2. The target compensation displacement ▽1+▽2 is determined.

[0110] In step 212, after controlling the screen to move to compensate for the target displacement, a screen assembly operation is performed on the middle frame and the screen.

[0111] The robotic arm is controlled to move ▽1+▽2, precisely attaching the screen to the mid-frame of the device to be assembled. This avoids the problem of poor edge positioning accuracy in the mid-frame during the screen assembly station. By using positioning information containing high-precision edge data obtained at the dispensing station before dispensing, the accuracy of screen assembly is greatly improved.

[0112] In other embodiments, when the dispensing station and the screen assembly station are both at the same operating point, and the position of the middle frame does not move during the dispensing and screen assembly process, the above steps 204-208 can be replaced by the following steps: the screen assembly station determines the target positioning information based on the initial positioning information.

[0113] Since the position of the mid-frame remains unchanged, the initial positioning information collected by the dispensing station before glue application is still used in the screen assembly operation. Therefore, after obtaining the initial positioning information that indicates the path for the dispensing station to perform the dispensing operation, the dispensing station can send the initial positioning information to the screen assembly station to transmit the glue-picking path to the screen assembly equipment. The initial positioning information can be first position information, the movement path information of the robotic arm during the glue application operation, or other information that can indicate the position of the mid-frame edge. The screen assembly equipment determines the position of the mid-frame based on the dispensing path indicated by the initial positioning information, thereby performing the screen assembly operation. This method utilizes initial positioning information that is not affected by the glue application at the edge position during screen assembly, enabling accurate screen assembly.

[0114] This disclosure also provides a screen assembly system that can be applied to the screen assembly of electronic products on a pre-assembly production line in a smart factory. (Refer to...) Figure 3 A block diagram of a screen-grouping system according to an exemplary embodiment illustrates the system, the system comprising:

[0115] The first execution module 31 acquires the initial positioning information of the middle frame that has not undergone glue dispensing operation, and reports the initial positioning information to the wire control module.

[0116] The dispensing module 32 performs dispensing operations on the middle frame;

[0117] The wire control module 33 is used to receive the initial positioning information reported by the first execution module and to send the initial positioning information to the second execution module;

[0118] The second execution module 34 is used to obtain alternative positioning information of the middle frame that has undergone dispensing operation, and to determine the target positioning information of the middle frame based on the initial positioning information and the alternative positioning information sent by the wire control module.

[0119] The screen grouping module 35 performs a screen grouping operation on the middle frame and the screen based on the target positioning information.

[0120] by Figure 4Taking the assembly line scenario shown as an example, the first execution module 31 and the dispensing module 32 are located at the dispensing station. During the assembly of the mobile phone screen, at the dispensing station in front of the screen assembly station, a 3D camera scans and acquires the feature MARK point information and edge information of the current mobile phone frame, i.e., the initial positioning information, and uploads it to the wire control module 33. The dispensing module 32 performs high-precision dispensing based on the high-precision edge information of the frame in the initial positioning information. The first execution module 31 can also acquire the identification code SN of the mobile phone frame material. The first execution module 31 of the dispensing station packages the above data information and uploads the initial positioning information and identification code to the wire control module 33 after the dispensing production is completed.

[0121] The second execution module 34 and the screen assembly module 35 are located at the screen assembly station. At the screen assembly station, firstly, the mid-frame on the production line is scanned to obtain the material identification code (SN). The screen assembly station uploads the material SN to the wire control module 33 and obtains the corresponding initial positioning information: the edge information of the phone's mid-frame and the MARK point information of the phone's mid-frame. The second execution module 34 of the screen assembly station can perform calculations on the mid-frame edge information from the previous station. The screen assembly station uses the camera above the mid-frame to capture new phone mid-frame MARK point information, i.e., candidate positioning information. This data is then used to perform matrix calculations with the phone mid-frame MARK point information obtained from the wire control module 33 to obtain a transformation matrix M. This transformation matrix is ​​then used to convert the phone mid-frame edge information obtained from the wire control module 33 into the mid-frame edge information locally at the screen assembly station, i.e., the target positioning information. The second execution module 34 of the screen assembly station can perform high-precision compensation using the mid-frame edge information obtained from the previous station scan. The screen assembly station uses the converted edge information of the mid-frame to compare with the standard position to calculate the compensation value ▽1, and then adds the compensation value ▽2 obtained by visual positioning calculation of the touch screen. It controls the robotic arm that has grasped the touch screen to move ▽1+▽2, so as to accurately attach the touch screen to the mid-frame of the mobile phone.

[0122] In this embodiment, at the dispensing station preceding the screen assembly station, a higher-precision camera scans and acquires the MARK point information and edge information of the current mobile phone frame, and uploads it to the wired control module. During production, the wired control module transmits the frame edge information from the dispensing station to the screen assembly station. At the screen assembly station, through comparison mapping of the same MARK point, the accurate edge information of the same frame obtained at the previous station is mapped to this station. This screen assembly method locates the frame edge based on the dispensing closed-loop data from the previous station, avoiding the visual edge finding of the frame at the screen assembly station, which is prone to errors, thereby improving the screen assembly accuracy. By using the mapping of feature points that are not covered by dispensing for positioning, the problem of dispensing covering the real frame edge, causing the screen assembly equipment to be unable to recognize the edge, is avoided. The communication between the dispensing station and the assembly station is achieved through a wired control module, which solves the problem of data interaction between the front and rear stations. The assembly station can quickly reuse the mid-frame data of the dispensing station. Since this solution calls the edge data of the front station for mounting, the vision system of the assembly equipment used by the assembly station does not need to be as precise, thus reducing equipment costs.

[0123] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should know that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps may be performed in other orders or simultaneously.

[0124] Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by this disclosure.

[0125] Corresponding to the aforementioned application function implementation method embodiments, this disclosure also provides embodiments of application function implementation apparatus and corresponding terminals.

[0126] Reference Figure 5 A block diagram of a screen assembly device according to an exemplary embodiment shows that the device may include:

[0127] The initial positioning module 41 is used to obtain the initial positioning information of the middle frame before the glue dispensing operation;

[0128] Alternate positioning module 42 is used to obtain alternative positioning information of the middle frame that has undergone dispensing operation;

[0129] The target positioning module 43 is used to determine the target positioning information of the middle frame based on the initial positioning information and the alternative positioning information;

[0130] The screen grouping operation module 44 is used to perform screen grouping operation on the middle frame and the screen based on the target positioning information.

[0131] In one optional implementation, the initial positioning information includes at least one of the following:

[0132] First position information in a first coordinate system is used to indicate the edge of the middle frame that has not undergone adhesive dispensing.

[0133] Second position information in a first coordinate system for indicating feature points on the middle frame where no dispensing operation has been performed; and / or,

[0134] The alternative location information includes at least:

[0135] The third position information of the feature points on the middle frame that have undergone dispensing operation is used to indicate the third position information in the second coordinate system.

[0136] In one optional implementation, the initial positioning module 41 is specifically used for:

[0137] The initial positioning information is obtained by the dispensing station, and the initial positioning information is used to indicate the path for the dispensing station to perform the dispensing operation;

[0138] It is also used for: sending the initial positioning information to the screen assembly station by the dispensing station;

[0139] The target positioning module 43 is specifically used to: determine the target positioning information by the screen assembly station based on the initial positioning information.

[0140] In one optional implementation, when the mid-frame is conveyed to the screen assembly station via an assembly line after being glued at the glue dispensing station,

[0141] The alternative positioning module 42 is specifically used to: obtain the alternative positioning information by the screen assembly station in conjunction with the vision system;

[0142] The target positioning module 43 is specifically used to: determine the target positioning information by the screen assembly station based on the initial positioning information and the alternative positioning information.

[0143] In one optional implementation, the target positioning module 43 is specifically used for:

[0144] The screen assembly station determines a transformation matrix based on the second position information in the initial positioning information and the third position information in the alternative positioning information; wherein, the second position information is used to represent the position information of feature points on the middle frame that have not undergone glue dispensing in the first coordinate system; the third position information is used to represent the position information of feature points on the middle frame that have undergone glue dispensing in the second coordinate system; the transformation matrix is ​​used to indicate the mapping relationship between the first coordinate system and the second coordinate system;

[0145] Based on the transformation matrix, the first position information in the first coordinate system of the initial positioning information is transformed to the second coordinate system to obtain the fourth position information of the middle frame; the first position information is the position information of the edge of the middle frame without dispensing operation in the first coordinate system corresponding to the dispensing station;

[0146] The fourth location information is determined as the target location information.

[0147] In one optional implementation, the screen grouping operation module 44 is specifically used for:

[0148] Based on the target positioning information, the target compensation displacement that the screen needs to move is determined;

[0149] After controlling the screen to move to compensate for the target displacement, a screen assembly operation is performed on the middle frame and the screen.

[0150] In an optional implementation, when the screen grouping operation module 44 is used to determine the compensation displacement based on the target positioning information, it is specifically used for:

[0151] Based on the difference between the target positioning information and the preset positioning information of the middle frame, the first compensation displacement is calculated;

[0152] The second compensation displacement is calculated based on the difference between the screen positioning information and the preset positioning information of the screen.

[0153] The target compensation displacement is determined based on the first compensation displacement and the second compensation displacement.

[0154] In an optional implementation, the target positioning module 43 is further configured to:

[0155] Obtain the target identification information of the middle frame that has undergone dispensing operation;

[0156] The initial positioning module 41 is specifically used for:

[0157] Based on the pre-determined association between different mid-frame identification information and different initial positioning information, the initial positioning information corresponding to the target identification information is obtained.

[0158] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0159] Accordingly, in one aspect, this disclosure provides a terminal device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the screen assembly method described above.

[0160] Figure 6 This is a schematic diagram illustrating the structure of a terminal device 500 according to an exemplary embodiment. For example, the terminal device 500 can be a user device, specifically a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, wearable device such as smartwatch, smart glasses, smart bracelet, smart running shoes, etc.

[0161] Reference Figure 5 The terminal device 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.

[0162] Processing component 502 typically controls the overall operation of terminal device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0163] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of this data include instructions for any application or method operating on terminal device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0164] Power supply component 506 provides power to various components of terminal device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to terminal device 500.

[0165] Multimedia component 508 includes a screen that provides an output interface between the terminal device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When the device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0166] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when terminal device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0167] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0168] Sensor assembly 514 includes one or more sensors for providing status assessments of various aspects of terminal device 500. For example, sensor assembly 514 can detect the on / off state of device 500, the relative positioning of components such as the aforementioned components (e.g., the display and keypad of terminal device 500), changes in position of terminal device 500 or a component thereof, the presence or absence of user contact with terminal device 500, orientation or acceleration / deceleration of terminal device 500, and temperature changes of terminal device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0169] Communication component 516 is configured to facilitate wired or wireless communication between terminal device 500 and other devices. Terminal device 500 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the aforementioned communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0170] In an exemplary embodiment, the terminal device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0171] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 504 including instructions, which, when executed by the processor 520 of the terminal device 500, enables the terminal device 500 to perform the screen assembly method described above.

[0172] The non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0173] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0174] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for configuring screens, characterized in that, The method includes: Obtain the initial positioning information of the middle frame before glue dispensing; Obtain alternative positioning information for the middle frame that has undergone dispensing operation; Based on the initial positioning information and the alternative positioning information, the target positioning information of the middle frame is determined; Based on the target positioning information, a screen assembly operation is performed on the mid-frame and the screen; Determining the target positioning information of the mid-frame based on the initial positioning information and the alternative positioning information includes: The screen assembly station determines a transformation matrix based on the second position information in the initial positioning information and the third position information in the alternative positioning information; wherein, the second position information is used to represent the position information of the feature points on the middle frame that have not undergone glue dispensing in the first coordinate system; the third position information is used to represent the position information of the feature points on the middle frame that have undergone glue dispensing in the second coordinate system; the transformation matrix is ​​used to indicate the mapping relationship between the first coordinate system and the second coordinate system; Based on the transformation matrix, the first position information of the first coordinate system in the initial positioning information is transformed to the second coordinate system to obtain the fourth position information of the middle frame; the first position information is the position information of the edge of the middle frame without dispensing operation in the first coordinate system corresponding to the dispensing station; The fourth location information is determined as the target location information.

2. The method according to claim 1, characterized in that, The initial location information includes at least one of the following: First position information in a first coordinate system is used to indicate the edge of the middle frame that has not undergone adhesive dispensing. Second position information of feature points on the middle frame in the first coordinate system, used to indicate the position of feature points in the first coordinate system when no glue dispensing operation has been performed; And / or, The alternative location information includes at least: The third position information of the feature points on the middle frame that have undergone dispensing operation is used to indicate the third position information in the second coordinate system.

3. The method according to claim 1, characterized in that, The process of obtaining the initial positioning information of the mid-frame before adhesive dispensing includes: The initial positioning information is obtained by the dispensing station, and the initial positioning information is used to indicate the path for the dispensing station to perform the dispensing operation; The method further includes: The initial positioning information is sent from the dispensing station to the screen assembly station.

4. The method according to claim 3, characterized in that, After the middle frame undergoes the glue dispensing operation at the glue dispensing station, it is conveyed to the screen assembly station in an assembly line manner. The step of obtaining the alternative positioning information of the middle frame that has undergone dispensing includes: The alternative positioning information is obtained by the screen assembly station in conjunction with the vision system.

5. The method according to claim 1, characterized in that, The step of performing a screen assembly operation on the mid-frame and the screen based on the target positioning information includes: Based on the target positioning information, the target compensation displacement that the screen needs to move is determined; After controlling the screen to move to compensate for the target displacement, a screen assembly operation is performed on the middle frame and the screen.

6. The method according to claim 5, characterized in that, The step of determining the compensation displacement based on the target positioning information includes: Based on the difference between the target positioning information and the preset positioning information of the middle frame, the first compensation displacement is calculated; The second compensation displacement is calculated based on the difference between the screen positioning information and the preset positioning information of the screen. The target compensation displacement is determined based on the first compensation displacement and the second compensation displacement.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the target identification information of the middle frame that has undergone dispensing operation; The process of obtaining the initial positioning information of the mid-frame before adhesive dispensing includes: Based on the pre-determined association between different mid-frame identification information and different initial positioning information, the initial positioning information corresponding to the target identification information is obtained.

8. A screen assembly device, characterized in that, The device includes: The initial positioning module is used to obtain the initial positioning information of the middle frame before the glue dispensing operation; Alternate positioning module, used to obtain alternative positioning information of the middle frame that has undergone dispensing operation; The target positioning module is used to determine the target positioning information of the middle frame based on the initial positioning information and the alternative positioning information; The screen grouping operation module is used to perform screen grouping operation on the middle frame and the screen based on the target positioning information; The target localization module is also used for: The screen assembly station determines a transformation matrix based on the second position information in the initial positioning information and the third position information in the alternative positioning information; wherein, the second position information is used to represent the position information of the feature points on the middle frame that have not undergone glue dispensing in the first coordinate system; the third position information is used to represent the position information of the feature points on the middle frame that have undergone glue dispensing in the second coordinate system; the transformation matrix is ​​used to indicate the mapping relationship between the first coordinate system and the second coordinate system; Based on the transformation matrix, the first position information of the first coordinate system in the initial positioning information is transformed to the second coordinate system to obtain the fourth position information of the middle frame; the first position information is the position information of the edge of the middle frame without dispensing operation in the first coordinate system corresponding to the dispensing station; The fourth location information is determined as the target location information.

9. A screen grouping system, characterized in that, The system includes: The first execution module obtains the initial positioning information of the middle frame before the glue dispensing operation, and reports the initial positioning information to the wire control module. The dispensing module performs dispensing operations on the middle frame. The wire control module is used to receive the initial positioning information reported by the first execution module, and to send the initial positioning information to the second execution module; The second execution module is used to acquire alternative positioning information of the middle frame that has undergone dispensing operation, and to determine the target positioning information of the middle frame based on the initial positioning information and the alternative positioning information sent by the wire control module. The screen grouping module performs screen grouping operations on the mid-frame and the screen based on the target positioning information; The second execution module is also used for: The screen assembly station determines a transformation matrix based on the second position information in the initial positioning information and the third position information in the alternative positioning information; wherein, the second position information is used to represent the position information of the feature points on the middle frame that have not undergone glue dispensing in the first coordinate system; the third position information is used to represent the position information of the feature points on the middle frame that have undergone glue dispensing in the second coordinate system; the transformation matrix is ​​used to indicate the mapping relationship between the first coordinate system and the second coordinate system; Based on the transformation matrix, the first position information of the first coordinate system in the initial positioning information is transformed to the second coordinate system to obtain the fourth position information of the middle frame; the first position information is the position information of the edge of the middle frame without dispensing operation in the first coordinate system corresponding to the dispensing station; The fourth location information is determined as the target location information.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the screen assembly method according to any one of claims 1 to 7.

11. A terminal device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the steps of the screen assembly method according to any one of claims 1 to 7.

Images