Installation state determination method and device, terminal and computer readable storage medium

Abstract

The invention provides an installation state determination method and device, a terminal and a computer readable storage medium. The installation state determination method comprises the steps that a force signal data set generated when an installation part is subjected to external force in the current time period is acquired; the force signal data set comprises detection data corresponding to multiple dimensions in the current time period; performing dimension splicing on the detection data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set; and based on the multi-dimensional splicing data of the force signal data set, determining whether the installation of the installation part is completed or not. According to the method and the device, the detection data of each dimension is subjected to dimension splicing to obtain the multi-dimensional splicing data corresponding to the force signal data set, and the stress change curve corresponding to the multi-dimensional splicing data corresponding to the force signal data set is compared with the stress change curve corresponding to the completed installation, so that whether the installation of the installation part is completed or not is determined; and the installation state detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic assembly equipment for accessories, and particularly to a method, device, terminal and computer-readable storage medium for determining an installation state. Background Art

[0002] There are many snap connections in both the fields of automobile assembly and battery assembly. In production, snaps are mainly installed manually or by robots. The efficiency of manual installation is difficult to meet the current production needs, and there are problems of incomplete installation. Summary of the Invention

[0003] The main technical problem to be solved by the present invention is to provide a method, device, terminal and computer-readable storage medium for determining an installation state, which can accurately determine the installation state of a snap.

[0004] In a first aspect, the present application provides a method for determining an installation state. The method for determining an installation state includes: obtaining a force signal data set generated when an installation part is subjected to an external force during a current period; the force signal data set includes detection data corresponding to multiple dimensions within the current period; performing dimension splicing on the detection data of each dimension to obtain multi-dimensional splicing data corresponding to the force signal data set; and determining whether the installation of the installation part is completed based on the multi-dimensional splicing data of the force signal data set.

[0005] In the technical solution of the embodiment of the present application, by performing dimension splicing on the detection data of each dimension to obtain multi-dimensional splicing data corresponding to the force signal data set, comparing the force change curve corresponding to the multi-dimensional splicing data corresponding to the force signal data set with the force change curve corresponding to the completion of installation, and then determining whether the installation of the installation part is completed, the accuracy of installation state detection is improved.

[0006] In some embodiments, determining whether the installation of the installation part is completed based on the multi-dimensional splicing data of the force signal data set includes: extracting features from the multi-dimensional splicing data to obtain feature information of the multi-dimensional splicing data; performing feature aggregation on the feature information of the multi-dimensional splicing data to obtain fused feature information; and comparing the fused feature information with preset feature information in a database to determine whether the installation of the installation part is completed.

[0007] In the technical solution of the embodiment of the present application, by extracting features from the multi-dimensional splicing data and fusing the feature information of six dimensions corresponding to each moment in each multi-dimensional splicing data to obtain the fused feature information of the force signal data set, comparing the fused feature information with the preset feature information to determine whether the installation part is installed, and then facilitating the detection of the installation state of the installation part.

[0008] In some embodiments, determining whether the installation of the installation part is completed based on the multi-dimensional spliced data of the force signal dataset includes: inputting the multi-dimensional spliced data of the force signal dataset into a deep neural network; the deep neural network includes a feature extraction network and an aggregation network; using the feature extraction network to extract features from the multi-dimensional spliced data to obtain the feature information of the multi-dimensional spliced data; using the aggregation network to perform feature aggregation and recognition on the feature information of the multi-dimensional spliced data to determine whether the installation of the installation part is completed.

[0009] In the technical solution of the embodiment of the present application, through the sequential connection of the feature extraction network and the aggregation network in the deep neural network, using an end-to-end deep learning algorithm, it is determined whether the installation of the installation part is completed based on the multi-dimensional spliced data, which is convenient for detecting the installation state and improving the detection accuracy.

[0010] In some embodiments, the external force includes pressure acting in multiple stages; splicing the detection data of each dimension to obtain the multi-dimensional spliced data corresponding to the force signal dataset, including: extracting the target data corresponding to each stage from the detection data corresponding to the dimension based on the force characteristics corresponding to each stage; the target data is associated with time information; splicing the target data of each stage corresponding to the dimension based on the time information of the target data to obtain the force trajectory data of the dimension; sequentially splicing the force trajectory data of each dimension to obtain the multi-dimensional spliced data corresponding to the force signal dataset.

[0011] In the technical solution of the embodiment of the present application, according to the force characteristics corresponding to each stage, the target data corresponding to each stage is screened out from the detection data corresponding to the dimension; by splicing the target data corresponding to the dimension and splicing the force trajectory data of each dimension, the multi-dimensional spliced data corresponding to the force signal dataset is obtained, eliminating invalid data, reducing the calculation amount, and improving the data accuracy.

[0012] In some embodiments, obtaining the force signal dataset generated when the installation part is subjected to an external force in the current period includes: at least collecting the force information of the third coordinate axis generated when the installation part is subjected to an external force at the current moment, and the torque information corresponding to the first coordinate axis and the second coordinate axis respectively; the first coordinate axis, the second coordinate axis, and the third coordinate axis are perpendicular to each other and intersect at the center point of the installation part; in response to the force information corresponding to the third coordinate axis not exceeding the first preset value, the force information and torque information collected at the current moment are attributed to the force signal dataset corresponding to the current period, and the third coordinate axis is parallel to the force direction of the installation part.

[0013] In the technical solution of the embodiment of the present application, by comparing the force information corresponding to the third coordinate axis with the first preset value, it is further determined whether the installation part is aligned with the installation position, improving the matching accuracy of the installation part and reducing the situation of the installation part being damaged by force.

[0014] In some embodiments, the state determination method further includes: in response to the force information corresponding to the third coordinate axis exceeding a first preset value, determining that the mounting member is misaligned with the mounting position; comparing the torque information corresponding to the first coordinate axis and the second coordinate axis with a second preset value respectively to determine whether it is necessary to adjust the pose of the mounting member.

[0015] In the technical solution of the embodiment of the present application, by comparing the torque information corresponding to the first coordinate axis and the second coordinate axis with the second preset value respectively, the offset degree between the mounting member and the mounting position is further determined, and then whether to adjust the pose of the mounting member is determined, reducing the risk of damage to the mounting member caused by misalignment of the mounting member.

[0016] In some embodiments, comparing the torque information corresponding to the first coordinate axis and the second coordinate axis with the second preset value respectively to determine whether it is necessary to adjust the pose of the mounting member includes: in response to at least one of the torque information corresponding to the first coordinate axis and the second coordinate axis being greater than the second preset value, determining that the mounting member is partially aligned with the mounting position; adjusting the pose of the mounting member based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively to align the mounting member with the mounting position.

[0017] In the technical solution of the embodiment of the present application, when the offset degree between the mounting member and the mounting position is small, the position of the mounting member can be automatically adjusted, reducing the risk of damage to the mounting member caused by misalignment of the mounting member.

[0018] In some embodiments, adjusting the pose of the mounting member based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively includes: in response to the direction of the torque information corresponding to the first coordinate axis being clockwise, adjusting the mounting member to move in the positive direction of the second coordinate axis; or, in response to the direction of the torque information corresponding to the first coordinate axis being counterclockwise, adjusting the mounting member to move in the opposite direction of the second coordinate axis.

[0019] In the technical solution of the embodiment of the present application, based on the direction of the torque information corresponding to the first coordinate axis, the adjustment position of the mounting member can be accurately determined, reducing the adjustment duration of the mounting member and saving the installation time of the mounting member.

[0020] In some embodiments, adjusting the pose of the mounting member based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively includes: in response to the direction of the torque information corresponding to the second coordinate axis being clockwise, adjusting the mounting member to move in the positive direction of the first coordinate axis; or, in response to the direction of the torque information corresponding to the second coordinate axis being counterclockwise, adjusting the mounting member to move in the opposite direction of the first coordinate axis.

[0021] In the technical solution of the embodiment of the present application, based on the direction of the torque information corresponding to the second coordinate axis, the adjustment position of the installation part can be accurately determined, so as to facilitate the adjustment of the installation part to move on the plane, reduce the adjustment duration of the installation part, and save the installation time of the installation part.

[0022] In some embodiments, in response to at least one of the torque information corresponding to the first coordinate axis and the second coordinate axis being greater than a second preset value, after the step of determining that the installation part is partially aligned with the installation position, the method further includes: counting the number of consecutive times that the installation part is partially aligned with the installation position; and in response to the number of consecutive times reaching a preset number, starting an alarm.

[0023] In the technical solution of the embodiment of the present application, when the number of consecutive times that the installation part is partially aligned with the installation position exceeds the preset number, an alarm is started, so as to facilitate the operator to calibrate the robotic arm controlling the installation part and improve the alignment accuracy.

[0024] In some embodiments, comparing the torque information corresponding to the first coordinate axis and the second coordinate axis with the second preset value respectively to determine whether the pose of the installation part needs to be adjusted includes: in response to the torque information corresponding to the first coordinate axis and the second coordinate axis not exceeding the second preset value respectively, determining that the installation part is completely misaligned with the installation position; and controlling the installation part to stop installation and / or starting an alarm.

[0025] In the technical solution of the embodiment of the present application, when the deviation degree between the installation part and the installation position is large, it is not necessary to adjust the position of the installation part, and the installation part is controlled to stop installation, so as to reduce the risk of damage to the installation part caused by the misalignment of the installation part.

[0026] In a second aspect, the present application provides an installation state determination device, including: an acquisition module, configured to acquire a force signal data set generated when an external force acts on an installation part in a current period; the force signal data set includes detection data corresponding to multiple dimensions in the current period; a processing module, configured to splice the detection data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set; and a determination module, configured to determine whether the installation of the installation part is completed based on the multi-dimensional spliced data of the force signal data set.

[0027] In a third aspect, the present application provides a terminal, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor is configured to execute program data to implement the steps in the installation state determination method as described above.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the installation state determination method as described above are implemented.

[0029] It is understandable that the beneficial effects of the second to fourth aspects described above can be referred to the relevant descriptions in the first aspect above, and will not be elaborated here.

[0030] The above description is only an overview of the technical solution of the present application. In order to better understand the technical means of the present application, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specific embodiments of the present application are given. Brief Description of the Drawings

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0032] Figure 1 It is a schematic diagram of the force curve during the buckle installation process provided by the present application;

[0033] Figure 2 It is a schematic diagram of the structure during the buckle installation process provided by the present application;

[0034] Figure 3 It is a schematic flowchart of the installation state determination method provided by the present application;

[0035] Figure 4 It is a schematic flowchart of a specific embodiment of the installation state determination method provided by the present application;

[0036] Figure 5 is Figure 3 A schematic flowchart of step S2 in the installation state determination method provided;

[0037] Figure 6 It is a schematic framework diagram of an embodiment of the installation state determination device provided by the present application;

[0038] Figure 7 It is a schematic framework diagram of an embodiment of the terminal provided by the present application;

[0039] Figure 8 It is a schematic framework diagram of an embodiment of the computer-readable storage medium provided by the present application. Detailed Embodiments

[0040] The following will describe in detail the embodiments of the technical solution of the present application with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, so they are only examples and cannot be used to limit the protection scope of the present application.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs; the terms used herein are for the purpose of describing specific embodiments only and are not intended to limit this application; the terms "comprising" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion.

[0042] In the description of the embodiments of this application, technical terms such as "first" and "second" are only used to distinguish different objects and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity, specific order, or primary-secondary relationship of the indicated technical features. In the description of the embodiments of this application, "a plurality of" means more than two, unless otherwise specifically defined.

[0043] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0044] In the description of the embodiments of this application, the term "and / or" is merely a description of the associated relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after.

[0045] In the description of the embodiments of this application, the term "a plurality of" refers to more than two (including two). Similarly, "a plurality of groups" refers to more than two groups (including two groups), and "a plurality of pieces" refers to more than two pieces (including two pieces).

[0046] In the description of the embodiments of this application, technical terms such as "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the embodiments of this application and simplifying the description, and does not indicate or imply that the indicated device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of this application.

[0047] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installation", "connection", "linkage", "fixation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can also be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0048] In the existing robot installation buckle process, first, the suction nozzle of the pressing mechanism is used to suck and fix the buckle or the robot arm is used to grab the buckle. Then, the installation position of the buckle is located through the positioning module. Next, after moving the buckle to the installation position, the buckle is pressed down by the pressing cylinder to complete the installation of the buckle. Finally, the recognition probe is used to detect whether there are situations such as missing installation, damage, or improper installation of the buckle.

[0049] The current technical solution highly depends on the accuracy of the positioning module and the accuracy of the repeated movement of the robot. Once one of them has a problem, the buckle cannot be installed in place.

[0050] The buckle automatic assembly system provided in this embodiment includes: a robot arm fixed base for providing sufficient fixing force for the robot arm; a robot arm body; a sensor; and a translational mechanical gripper for grasping the buckle.

[0051] The translational mechanical gripper is used to grasp and fix the buckle. The fingertips of the translational mechanical gripper can be easily replaced. The fingertips of the translational mechanical gripper remain completely parallel throughout the operation process, and the production error of such mechanical grippers is very low, enabling high-precision tasks. It can enable the robot to precisely grasp objects within a very small size range. The translational mechanical gripper is used to move the clip to the installation position of the clip hole.

[0052] The sensor provided in this embodiment is a silicon strain gauge force sensor, which is more excellent than other types of sensors in terms of stability, signal-to-noise ratio, and dynamic characteristics, and has a higher stiffness. Installing the sensor 3 at the end of the translational mechanical gripper can measure the forces and torques borne when the end effector of the robot comes into contact with the external environment or grasps the workpiece, capture the signals of the forces and torques in different directions of X, Y, and Z of the translational mechanical gripper, and use them as the force sensing information input for the force control and motion control of the robot arm body. The six-axis collaborative robot used for the robot arm body generally has 6 degrees of freedom, including rotation (S axis), lower arm (L axis), upper arm (U axis), wrist rotation (R axis), wrist swing (B axis), and wrist rotation (T axis). The 6 joints synthesize to achieve the 6-degree-of-freedom movement of the end, with many advantages such as high flexibility, large load capacity, and high positioning accuracy.

[0053] The snap installation is part of a flexible manufacturing system. The greatest feature of a six-axis industrial robot is its flexible start-up. It is an important component of a flexible manufacturing system and can perform flexible movements like a human in automatic assembly work.

[0054] By installing sensors at the end of the robotic arm to analyze and calculate the forces on the snap, it is then determined whether the snap is accurately aligned and installed. When the snap is not fully aligned with the installation position, the robotic arm is controlled based on the signal characteristics collected by the sensors to move the snap to a position aligned with the installation position, and then the snap is installed at the installation position.

[0055] Please refer to Figure 1 and Figure 2 , Figure 1 which is a schematic diagram of the force curve during the snap installation process provided by this application; Figure 2 which is a schematic structural diagram of the snap installation process provided by this application.

[0056] The snap is an elastic object. The mechanical characteristics during the process of the snap moving from a suspended state to being inserted into the card hole of the installation position are regular, and the signals obtained by the sensors are also regular. The forces / moments involved in the snap are complex and are the result of the combined action of multiple forces / moments. However, the forces involved in the snap installation process are basically regular. Therefore, only the force in the F z direction is used here to schematically illustrate the entire process.

[0057] During the period from 0 - t0, the snap is in a suspended state at the beginning of the installation. At this time, the value of the F z force is always very small and is close to 0 ideally; during the period from t0 - t1, when the snap continues to move downward until it contacts the inclined plane and starts to be pressed down, since the elastic force of the snap gradually increases, the frictional force also gradually increases accordingly. The value of the F z force will increase regularly, as shown in Figure 2 (a); during the period from t1 - t2, when the snap moves past the inclined plane and continues to be pressed down, since the snap basically does not deform and the frictional force is constant, the value of the F z force basically does not change, as shown in Figure 2 (b); during the period from t2 - t3, when the snap just snaps on, since the snap resumes deformation and the elastic force quickly becomes smaller, the generated frictional force also quickly becomes smaller. Therefore, the value of the F z force will suddenly become smaller in a short period; during the period from t3 - t4, due to the elasticity of the snap, there will be a short-time oscillation after it snaps on. The value of the F z force will also oscillate within a short period with an amplitude that becomes smaller and smaller and finally tends to be stable, as shown in Figure 2 (c).

[0058] The shape of the same type of buckle is fixed, and the curve generated by the same type of buckle is of the same type. Therefore, it is possible to determine whether the buckle is installed completely by calculating and analyzing the state data of the entire process from when the sensor just touches the part to when the elastic force of the buckle tends to be stable.

[0059] Please refer to Figure 3 and Figure 4 , Figure 3 which is a schematic flowchart of the installation state determination method provided by this application; Figure 4 which is a schematic flowchart of a specific embodiment of the installation state determination method provided by this application.

[0060] In this embodiment, an installation state determination method is provided. The method embodiments provided in the embodiments of this application can be executed in a mobile terminal, a computer terminal, or a similar computing device. The installation state determination method includes the following steps.

[0061] S1: Obtain a force signal data set generated when the installation part is subjected to an external force during the current period; the force signal data set includes detection data corresponding to multiple dimensions during the current period.

[0062] S2: Concatenate the detection data of each dimension to obtain multi-dimensional concatenated data corresponding to the force signal data set.

[0063] S3: Determine whether the installation of the installation part is completed based on the multi-dimensional concatenated data of the force signal data set.

[0064] In the technical solution of the embodiments of this application, by concatenating the detection data of each dimension to obtain multi-dimensional concatenated data corresponding to the force signal data set, and comparing the force change curve corresponding to the multi-dimensional concatenated data of the force signal data set with the force change curve corresponding to the completed installation. Furthermore, it is determined whether the installation of the installation part is completed, improving the accuracy of the installation state detection.

[0065] In some embodiments, the steps of obtaining the force signal data set generated when the installation part is subjected to an external force during the current period are as follows.

[0066] Specifically, after the robotic arm grabs and fixes the installation part, the robotic arm controls the installation part to move vertically downward, and at least collects the force information of the third coordinate axis generated when the installation part is subjected to an external force at the current moment, and the moment information corresponding to the first coordinate axis and the second coordinate axis respectively; the first coordinate axis, the second coordinate axis, and the third coordinate axis are perpendicular to each other and intersect at the center point of an installation part.

[0067] In some embodiments, a six-axis force sensor can be used to simultaneously collect the force information (F x , F y, F z ), and moment information (M x , M y , M z ). Extract the required force information of the third coordinate axis and the moment information corresponding to the first and second coordinate axes respectively from the force information (F x , F y , F z ) and moment information (M x , M y , M z ). By collecting the force information and moment information at each moment in the current period through a six - axis force sensor, the delay of data feedback can be reduced.

[0068] In another embodiment, the force information (F z ) of the third coordinate axis of the mounting member and the moment information (M x , M y ) corresponding to the first and second coordinate axes can also be collected separately by three independent sensors.

[0069] In still another embodiment, on the basis of collecting the force information (F z ) of the third coordinate axis of the mounting member and the moment information (M x , M y ) corresponding to the first and second coordinate axes, at least one of the force information (F x ) corresponding to the first coordinate axis, the force information (F y ) corresponding to the second coordinate axis, and the moment information (M z ) corresponding to the third coordinate axis can be collected by other independent sensors.

[0070] In some embodiments, the force information (F z ) of the third coordinate axis generated when the mounting member is subjected to an external force at the current moment and the moment information (M x , M y ) corresponding to the first and second coordinate axes can also be collected simultaneously by other sensors.

[0071] In other embodiments, the force information (F z ) of the third coordinate axis generated when the mounting member is subjected to an external force at the current moment, the moment information (M x , M y ) corresponding to the first and second coordinate axes, and the force information (F x ) corresponding to the first coordinate axis, the force information (F y ) corresponding to the second coordinate axis, and the moment information (Mz ) or at least one of them.

[0072] After the positioning module and the robotic arm have gone through multiple repeated operations, errors will continuously accumulate, sometimes resulting in the snap not being perfectly aligned with the installation position. Therefore, based on the force information corresponding to the third coordinate axis, it is determined whether the snap is aligned with the installation position.

[0073] In response to the force information corresponding to the preset coordinate axis not exceeding the first preset value, the force information and torque information collected at the current moment are attributed to the force signal data set corresponding to the current time period. The preset coordinate axis is parallel to the force direction of the installation part. The first preset value is calculated based on the material and shape of the card hole that matches the installation part and is set based on the installation position.

[0074] In this embodiment, the first coordinate axis is the X-axis, the second coordinate axis is the Y-axis, and the third coordinate axis is the Z-axis. The force direction of the installation part is parallel to the Z-axis direction.

[0075] Specifically, when the force information F corresponding to the Z-axis z does not exceed the first preset value, it indicates that the snap grasped by the robotic arm is aligned with the installation position. Subsequently, the installation part can be directly pressed to install the installation part at the installation position, thereby completing the installation of the installation part.

[0076] The force information and torque information collected during the stage from when the installation part is grasped and fixed by the sensor to when the force information corresponding to the third coordinate axis first exceeds the first preset value are intercepted, and the intercepted data is used as the force signal data set for the current time period.

[0077] In this embodiment, the force signal data set for the current time period includes the detection data corresponding to the six dimensions of the force information corresponding to the first coordinate axis, the second coordinate axis, and the third coordinate axis respectively, and the torque information rotating along the corresponding coordinate axes. The detection data includes the data corresponding to multiple moments respectively. For example, the detection data corresponding to the six dimensions are the signal curves corresponding to the six dimensions respectively.

[0078] In the technical solution of the embodiment of the present application, by comparing the force information corresponding to the third coordinate axis with the first preset value, it is further determined whether the installation part is aligned with the installation position, improving the matching accuracy of the installation part and reducing the situation of the installation part being damaged by force.

[0079] After the positioning module and the robotic arm have gone through multiple repeated operations, errors will continuously accumulate, sometimes resulting in the snap not being perfectly aligned with the installation position.

[0080] Specifically, in response to the force information corresponding to the third coordinate axis exceeding the first preset value, it is determined that the installation part is misaligned with the installation position; the torque information corresponding to the first coordinate axis and the second coordinate axis is compared with the second preset value respectively to determine whether the pose of the installation part needs to be adjusted. Specifically, the torque information corresponding to the first coordinate axis and the second coordinate axis is compared with the second preset value respectively to determine whether the installation part is aligned with the card hole at the installation position in the horizontal direction. Among them, the second preset value is calculated based on the material and shape of the card hole that matches the installation part set at the installation position. The second preset value may be the same as or different from the first preset value.

[0081] In response to at least one of the torque information corresponding to the first coordinate axis and the second coordinate axis being greater than the second preset value, it is determined that the installation part is partially aligned with the installation position; the pose of the installation part is adjusted based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively, so that the installation part is aligned with the installation position. Specifically, the force information corresponding to the third coordinate axis during the process of adjusting the installation part is detected in real time. When the force information corresponding to the third coordinate axis does not exceed the first preset value, it indicates that the installation part is aligned with the card hole at the installation position.

[0082] In response to the torque information corresponding to the first coordinate axis and the second coordinate axis not exceeding the second preset value respectively, it is determined that the installation part is completely misaligned with the installation position, that is, completely displaced, and then the installation of the installation part is controlled to stop and / or an alarm is started. The robotic arm can be controlled to stop moving and an alarm can be issued to stop the installation of the installation part, reducing the damage to the buckle or card hole. And it is convenient to notify the operator that the cumulative error of the robotic arm and the positioning module is too large at this time, and the robotic arm controlling the installation of the installation part needs to be calibrated.

[0083] In the technical solution of the embodiment of the present application, by comparing the torque information corresponding to the first coordinate axis and the second coordinate axis with the second preset value respectively, the offset degree between the installation part and the installation position is further determined. Furthermore, when the offset degree between the installation part and the installation position is small, the position of the installation part can be automatically adjusted, improving the alignment accuracy and reducing the risk of damage to the installation part caused by misalignment of the installation part.

[0084] Specifically, the pose of the installation part is adjusted based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively.

[0085] In a specific embodiment, in response to the direction of the torque information corresponding to the first coordinate axis being clockwise, the installation part is adjusted to move along the positive direction of the second coordinate axis.

[0086] In a specific embodiment, in response to the direction of the torque information corresponding to the first coordinate axis being counterclockwise, the installation part is adjusted to move along the opposite direction of the second coordinate axis.

[0087] In a specific embodiment, in response to the direction of the torque information corresponding to the second coordinate axis being clockwise, the adjusting member is adjusted to move in the positive direction of the first coordinate axis.

[0088] In a specific embodiment, in response to the direction of the torque information corresponding to the second coordinate axis being counterclockwise, the adjusting member is adjusted to move in the opposite direction of the first coordinate axis.

[0089] In an embodiment, the first coordinate axis may be the X-axis, and the second coordinate axis may be the Y-axis. Specifically, when the torque information corresponding to the X-axis is positive, the adjusting member is adjusted to move in the positive direction of the Y-axis. Specifically, when the torque information corresponding to the X-axis is negative, the adjusting member is adjusted to move in the opposite direction of the Y-axis. Among them, the positive direction and the opposite direction are opposite. Specifically, when the torque information corresponding to the Y-axis is positive, the adjusting member is adjusted to move in the positive direction of the X-axis. Specifically, when the torque information corresponding to the Y-axis is negative, the adjusting member is adjusted to move in the opposite direction of the X-axis.

[0090] In another embodiment, the first coordinate axis may be the Y-axis, and the second coordinate axis may be the X-axis. Specifically, when the torque information corresponding to the Y-axis is positive, the adjusting member is adjusted to move in the positive direction of the X-axis. Specifically, when the torque information corresponding to the Y-axis is negative, the adjusting member is adjusted to move in the opposite direction of the X-axis. Among them, the positive direction and the opposite direction are opposite. Specifically, when the torque information corresponding to the X-axis is positive, the adjusting member is adjusted to move in the positive direction of the Y-axis. Specifically, when the torque information corresponding to the X-axis is negative, the adjusting member is adjusted to move in the opposite direction of the Y-axis.

[0091] In the technical solution of the embodiment of the present application, based on the directions of the torque information respectively corresponding to the first coordinate axis and the second coordinate axis, by controlling the adjusting member to move in any direction on the horizontal plane where it is located, the adjustment position of the adjusting member is accurately determined, the adjustment duration of the adjusting member is reduced, and the installation time of the adjusting member is saved.

[0092] In some embodiments, the steps of splicing the detection data of each dimension to obtain the multi-dimensional spliced data corresponding to the force signal dataset are as follows.

[0093] Please refer to Figure 5 , Figure 5 is Figure 3 a schematic flow chart of step S2 in the provided installation state determination method.

[0094] Specifically, the external force includes pressure acting in multiple stages.

[0095] S21: Based on the force characteristics corresponding to each stage, extract the target data corresponding to each stage from the detection data corresponding to the dimension; the target data is associated with time information.

[0096] In a specific embodiment, according to Figure 1 As shown, the process of snap installation includes four stages. The force characteristic in the first stage is that as time increases, the force gradually increases. The force characteristic in the second stage is that as time increases, the force drops sharply and tends to be stable. The force characteristic in the third stage is that as time increases, the force gradually decreases based on the force in the second stage. The force characteristic in the fourth stage is that as time increases, the force tends to be stable, and the force in the fourth stage is less than the force in the second stage.

[0097] Extract the target data corresponding to the first stage, second stage, third stage, and fourth stage respectively from the detection data corresponding to the dimension.

[0098] S22: Based on the time information of the target data, splice the target data of each stage corresponding to the dimension to obtain the force trajectory data of the dimension.

[0099] Specifically, splice the target data corresponding to the first stage, second stage, third stage, and fourth stage corresponding to the dimension in sequence to obtain the force trajectory data corresponding to each dimension. The force trajectory data can be a force change curve.

[0100] S23: Splice the force trajectory data of each dimension in sequence to obtain the multi-dimensional spliced data corresponding to the force signal dataset.

[0101] Specifically, splice the force trajectory data corresponding to the six dimensions in sequence to obtain the multi-dimensional spliced data corresponding to the force signal dataset. That is, the multi-dimensional spliced data is a one-dimensional force change curve.

[0102] In the technical solution of the embodiment of the present application, according to the force characteristics corresponding to each stage, the target data corresponding to each stage is screened out from the detection data corresponding to the dimension; by splicing the target data corresponding to the dimension and splicing the force trajectory data of each dimension, the multi-dimensional spliced data corresponding to the force signal dataset is obtained, eliminating invalid data, reducing the calculation amount, and improving the data accuracy.

[0103] In some embodiments, based on the multi-dimensional spliced data of the force signal dataset, the steps to determine whether the installation of the installation part is completed are as follows.

[0104] In one embodiment, feature extraction is performed on the multi-dimensional spliced data to obtain the feature information of the multi-dimensional spliced data. Feature aggregation is performed on the feature information of the multi-dimensional spliced data to obtain the fused feature information; the fused feature information is compared with the preset feature information in the database to determine whether the installation of the installation part is completed.

[0105] If the fused feature information matches any of the preset feature information in the database, it is determined that the installation part has been installed; if the fused feature information does not match all of the preset feature information in the database, it is determined that the installation part has not been installed.

[0106] In a specific embodiment, the multi-dimensional spliced data of the force signal dataset is input into a deep neural network; the deep neural network includes a feature extraction network and an aggregation network. The feature extraction network can be a Deep Residual Shrinkage Networks with Channel-shared Thresholds (DRSN-CS). DRSN-CS is a variant of Resnet that uses an adaptive soft threshold layer (residual shrinkage layer) to eliminate noise. The aggregation network can be a Vector of Locally Aggregated Descriptors (VLAD) network that constructs the aggregated local descriptors into a vector and uses this vector as the global descriptor of the image.

[0107] The deep neural network in this embodiment uses an end-to-end deep learning algorithm and trains the deep neural network using a triplet loss function. The training samples include target sample data, positive sample data, and negative sample data, all of which are multi-dimensional sample data. Among them, the labels of the positive sample data and the target sample data are the same; the labels of the negative sample data and the target sample data are different.

[0108] The feature extraction network is used to extract the feature information of the multi-dimensional spliced data, and the aggregation network is used to perform feature aggregation and recognition on the feature information of the multi-dimensional spliced data to determine whether the installation of the installation part is completed. Specifically, the aggregation network performs feature aggregation and recognition on the feature information of the multi-dimensional spliced data and outputs that the installation part is currently in a completed installation state or an uncompleted installation state.

[0109] In the technical solution of the embodiment of the present application, by extracting the features of the multi-dimensional spliced data and fusing the feature information of the six dimensions corresponding to each moment in each multi-dimensional spliced data, the fused feature information of the force signal dataset is obtained, and the fused feature information is compared with the preset feature information to determine whether the installation part is installed, thereby facilitating the detection of the installation state of the installation part.

[0110] Please refer to Figure 6 , Figure 6 which is a schematic framework diagram of an embodiment of the installation state determination device provided by the present application.

[0111] This embodiment provides an installation status determination device 60, which includes an acquisition module 61, a processing module 62, and a determination module 63.

[0112] The acquisition module 61 is configured to acquire a force signal data set generated when an external force acts on the installation part during the current period; the force signal data set includes detection data corresponding to multiple dimensions within the current period.

[0113] The processing module 62 is configured to splice the detection data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set.

[0114] The determination module 63 is configured to determine whether the installation of the installation part is completed based on the multi-dimensional spliced data of the force signal data set.

[0115] The installation status determination device provided in this embodiment splices the detection data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set, and compares the force change curve corresponding to the multi-dimensional spliced data of the force signal data set with the force change curve corresponding to the completed installation. Furthermore, it determines whether the installation of the installation part is completed, improving the accuracy of installation status detection.

[0116] Please refer to Figure 7 , Figure 7 which is a schematic framework diagram of an embodiment of the terminal provided in this application. The terminal 80 includes a mutually coupled memory 81 and a processor 82. The processor 82 is configured to execute program instructions stored in the memory 81 to implement the steps of any of the above-mentioned installation status determination method embodiments. In a specific implementation scenario, the terminal 80 may include, but is not limited to: a microcomputer, a server. In addition, the terminal 80 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0117] Specifically, the processor 82 is used to control itself and the memory 81 to implement the steps of any of the above-described method embodiments for determining the installation state. The processor 82 may also be referred to as a CPU (Central Processing Unit). The processor 82 may be an integrated circuit chip with the ability to process signals. The processor 82 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 82 may be implemented jointly by integrated circuit chips.

[0118] Please refer to Figure 8 , Figure 8 which is a schematic framework diagram of an embodiment of the computer-readable storage medium provided by this application. The computer-readable storage medium 90 stores program instructions 901 that can be run by a processor, and the program instructions 901 are used to implement the steps of any of the above-described method embodiments for determining the installation state.

[0119] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be elaborated here.

[0120] The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments. Their similarities or similarities can be referred to each other. For the sake of brevity, they will not be elaborated in this article.

[0121] In several embodiments provided by this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0122] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0123] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0124] The above are only the embodiments of the present invention, and do not limit the patent protection scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. An installation state determination method, characterized in that, The installation state determination method includes: Obtaining a force signal data set generated when the installation component is subjected to an external force during the current period; the force signal data set includes detection data corresponding to multiple dimensions within the current period; Performing dimension splicing on the detection data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set; Based on the multi-dimensional spliced data of the force signal data set, determining whether the installation of the installation component is completed.

2. The installation state determination method according to claim 1, characterized in that, The determining whether the installation of the installation component is completed based on the multi-dimensional spliced data of the force signal data set includes: Performing feature extraction on the multi-dimensional spliced data to obtain feature information of the multi-dimensional spliced data; Performing feature aggregation on the feature information of the multi-dimensional spliced data to obtain fused feature information; Comparing the fused feature information with preset feature information in the database to determine whether the installation of the installation component is completed.

3. The installation state determination method according to claim 1, characterized in that, The determining whether the installation of the installation component is completed based on the multi-dimensional spliced data of the force signal data set includes: Inputting the multi-dimensional spliced data of the force signal data set into a deep neural network; the deep neural network includes a feature extraction network and an aggregation network; Using the feature extraction network to perform feature extraction on the multi-dimensional spliced data to obtain feature information of the multi-dimensional spliced data; Using the aggregation network to perform feature aggregation and recognition on the feature information of the multi-dimensional spliced data to determine whether the installation of the installation component is completed.

4. The installation state determination method according to any one of claims 1 to 3, characterized in that, The external force includes pressure actions in multiple stages; The performing dimension splicing on the detection data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set includes: Based on the force characteristics corresponding to each stage, extracting target data corresponding to each stage from the detection data corresponding to the dimension; The target data is associated with time information; Based on the time information of the target data, splicing the target data of each stage corresponding to the dimension to obtain the force trajectory data of the dimension; Sequentially splicing the force trajectory data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set.

5. The installation state determination method according to any one of claims 1 to 3, characterized in that, The obtaining a force signal data set generated when the installation component is subjected to an external force during the current period includes: Collecting at least the force information of the third coordinate axis generated when the installation component is subjected to an external force at the current moment, and the moment information corresponding to the first coordinate axis and the second coordinate axis respectively; the first coordinate axis, the second coordinate axis, and the third coordinate axis are perpendicular to each other and intersect at the center point of the installation component; In response to the force information corresponding to the third coordinate axis not exceeding a first preset value, attributing the force information and the moment information collected at the current moment to the force signal data set corresponding to the current period, and the third coordinate axis is parallel to the force direction of the installation component.

6. The installation state determination method according to claim 5, characterized in that, The state determination method further includes: In response to the force information corresponding to the third coordinate axis exceeding the first preset value, determining that the installation component is not aligned with the installation position; Compare the torque information corresponding to the first coordinate axis and the second coordinate axis with a second preset value respectively to determine whether the pose of the mounting member needs to be adjusted.

7. The installation state determination method according to claim 6, characterized in that, The step of comparing the torque information corresponding to the first coordinate axis and the second coordinate axis with a second preset value respectively to determine whether the pose of the mounting member needs to be adjusted includes: In response to at least one of the torque information corresponding to the first coordinate axis and the second coordinate axis being greater than the second preset value, it is determined that the mounting member is partially aligned with the mounting position; Adjust the pose of the mounting member based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively, so that the mounting member is aligned with the mounting position.

8. The installation state determination method according to claim 7, characterized in that, The step of adjusting the pose of the mounting member based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively includes: In response to the direction of the torque information corresponding to the first coordinate axis being clockwise, adjust the mounting member to move along the positive direction of the second coordinate axis; Or, in response to the direction of the torque information corresponding to the first coordinate axis being counterclockwise, adjust the mounting member to move along the reverse direction of the second coordinate axis.

9. The installation state determination method according to claim 7 or 8, characterized in that, The step of adjusting the pose of the mounting member based on the torque information corresponding to the first coordinate axis and the second coordinate axis respectively includes: In response to the direction of the torque information corresponding to the second coordinate axis being clockwise, adjust the mounting member to move along the positive direction of the first coordinate axis; Or, in response to the direction of the torque information corresponding to the second coordinate axis being counterclockwise, adjust the mounting member to move along the reverse direction of the first coordinate axis.

10. The installation state determination method according to claim 7, characterized in that, After the step of determining that the mounting member is partially aligned with the mounting position in response to at least one of the torque information corresponding to the first coordinate axis and the second coordinate axis being greater than the second preset value, the method further includes: Count the consecutive times that the mounting member is partially aligned with the mounting position; In response to the consecutive times reaching a preset number of times, start an alarm.

11. The installation state determination method according to claim 6, characterized in that, The step of comparing the torque information corresponding to the first coordinate axis and the second coordinate axis with a second preset value respectively to determine whether the pose of the mounting member needs to be adjusted includes: In response to the torque information corresponding to the first coordinate axis and the second coordinate axis both not exceeding the second preset value, it is determined that the mounting member is completely misaligned with the mounting position; Control the mounting member to stop installation and / or start an alarm.

12. An installation state determination device, characterized in that, The installation state determination device includes: An acquisition module, configured to acquire a force signal data set generated when an external force acts on a mounting member in a current period; the force signal data set includes detection data corresponding to multiple dimensions in the current period; A processing module, configured to perform dimension splicing on the detection data of each dimension to obtain multi-dimensional spliced data corresponding to the force signal data set; A determination module, configured to determine whether the installation of the mounting member is completed based on the multi-dimensional spliced data of the force signal data set.

13. A terminal, characterized in that, The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor. The processor is configured to execute program data to implement the steps in the installation state determination method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements the steps in the installation state determination method according to any one of claims 1 to 11.

Images