Contact point detection method and device of contact pin electricity-taking connector based on optic nerves

Abstract

The invention provides a contact point detection method and a contact point detection device for a contact pin power-taking connector based on optic nerves, belongs to the technical field of image processing, and aims to ensure the robustness of detection. The method is applied to the electronic equipment and comprises the steps that the electronic equipment obtains at least two images, the at least two images are obtained by shooting a contact point of a pin electricity-taking connector in different shooting directions through shooting equipment, and each image in the at least two images comprises the contact point; and the electronic device associates the at least two images in the spatial dimension through an optic neural network model to obtain a processing result output by the optic neural network model, and the processing result indicates whether the contact point has poor contact.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for detecting contact points of a pin-type electrical connector based on visual nerves. Background Art

[0002] With the in-depth development of industrial automation and intelligent manufacturing, the performance and reliability of electrical equipment have become important factors affecting production efficiency and safety. As a key component of power supply equipment, the quality of the contact point of the pin power supply connector directly affects the stability of the equipment. At present, the traditional contact point detection method mainly relies on manual experience and equipment performance testing, but it is time-consuming to perform manual detection and it is difficult to meet the needs of large-scale production lines. Manual detection depends on the experience of the inspectors and is prone to misjudgment. Therefore, it is considered to adopt a neural network model for implementation. For example, the collected data is first preprocessed to extract key features to prepare for the training of the neural network model. Then, a suitable neural network architecture is selected to train and optimize the model to improve the accuracy and robustness of the detection. Finally, the trained model is deployed in the actual production environment to realize automated detection, and the performance of the model is continuously monitored to ensure the reliability of the detection results.

[0003] However, how to ensure the robustness of neural network models is a current research issue. Summary of the invention

[0004] The embodiments of the present application provide a method and device for detecting contact points of a pin-type electrical connector based on visual nerves, so as to ensure the robustness of the detection.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, a contact point detection method for a pin power connector based on visual nerves is provided, which is applied to electronic devices, and the method comprises: the electronic device acquires at least two images, and the at least two images are images obtained by a shooting device shooting the contact points of the pin power connector in different shooting directions, and each of the at least two images contains the contact points; the electronic device associates the at least two images in the spatial dimension through a visual neural network model to obtain a processing result output by the visual neural network model, and the processing result indicates whether there is a poor contact situation at the contact point.

[0007] Optionally, the pin power connector includes a base and a pin, the pin includes a first end and a second end, the first end of the pin is arranged on the base, and a contact point for drawing power arranged between the first end and the second end includes a first contact point and a second contact point, the first contact point and the second contact point are distributed on both sides of the pin, a first reference line connecting the first contact point and the second contact point is set, and a second reference line of the first end and the second end is set, and the first reference line and the second reference line are located in a reference plane; at least two images include a first image and a second image, the first image is an image obtained by a shooting device shooting the first contact point and the second contact point in a first shooting direction, the first shooting direction and the reference direction are at a first preset angle, and the first preset angle is not 0, the second image is an image obtained by a shooting device shooting the first contact point and the second contact point in a second shooting direction, the second shooting direction and the reference direction are at a second preset angle, and the second preset angle is not 0, and the reference direction is a direction perpendicular to the reference plane; the image of the first contact point in the first image is partially identical to the image of the first contact point in the second image; the image of the second contact point in the first image is partially identical to the image of the second contact point in the second image.

[0008] Optionally, the electronic device associates at least two images in the spatial dimension through the visual neural network model to obtain a processing result output by the visual neural network model, including: the electronic device convolves the first image through the convolution layer of the visual neural network model to obtain a first convolution vector set, then performs pooling processing on the first convolution vector set through the pooling layer of the visual neural network model to obtain a first pooling vector set, and finally inputs the first pooling vector set into the first fully connected layer of the visual neural network model through the first channel to obtain a first intermediate analysis result output by the first fully connected layer; the electronic device convolves the second image through the convolution layer of the visual neural network model to obtain a second convolution vector set, then performs pooling processing on the second convolution vector set through the pooling layer of the visual neural network model to obtain a second pooling vector set, and finally inputs the second pooling vector set into the second fully connected layer of the visual neural network model through the second channel to obtain a second intermediate analysis result output by the second fully connected layer; wherein the first fully connected layer and the second fully connected layer share some neurons to realize association processing in the spatial dimension; the electronic device processes the first intermediate analysis result and the second intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer.

[0009] Optionally, the neurons included in the first fully connected layer and the neurons included in the second fully connected layer form a honeycomb connected network structure, and the part of neurons shared by the first fully connected layer and the second fully connected layer are: neurons in the honeycomb connected network structure that are directly connected to the neurons in the first fully connected layer that are not shared by the second fully connected layer, and are also directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer; in the honeycomb connected network structure, the neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer.

[0010] Optionally, the visual neural network model is configured as follows: when the visual neural network model processes a first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated, and when the visual neural network model processes a second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are activated, and some neurons shared by the first fully connected layer and the second fully connected layer are always activated when the visual neural network model processes the first image and the second image; the first fully connected layer is configured as follows: a training image set obtained by using a shooting device to shoot contact points in a first shooting insertion direction is trained until convergence, and the second fully connected layer is configured as follows: a training image set obtained by using a shooting device to shoot contact points in a second shooting insertion direction is trained until convergence.

[0011] Optionally, the pin power connector includes a base and a pin, the pin includes a first end and a second end, the first end of the pin is arranged on the base, and a contact point for power extraction arranged between the first end and the second end includes a first contact point and a second contact point, the first contact point and the second contact point are distributed on both sides of the pin, a first reference line connecting the first contact point and the second contact point is set, and a second reference line of the first end and the second end is set, and the first reference line and the second reference line are located in a reference plane; at least two images include a first image, a second image and a third image, the first image is an image obtained by a shooting device shooting the first contact point and the second contact point in a first shooting direction, the first shooting direction is at a first preset angle to the reference direction, and the first preset angle is . Assume that the angle is not 0, the second image is an image obtained by the photographing device photographing the first contact point and the second contact point in a second photographing direction, and the second photographing direction is the same direction as the reference direction; the third image is an image obtained by the photographing device photographing the first contact point and the second contact point in a third photographing direction, and the third photographing direction and the reference direction are at a second preset angle, the second preset angle is not 0, and the reference direction is a direction perpendicular to the reference plane; the image of the first contact point in the first image, the image of the first contact point in the second image, and the image of the first contact point in the third image are partially the same; the image of the second contact point in the first image, the image of the second contact point in the second image, and the image of the second contact point in the third image are partially the same.

[0012] Optionally, the electronic device associates and processes at least two images in the spatial dimension through a visual neural network model to obtain a processing result output by the visual neural network model, including: the electronic device convolves the first image through the convolution layer of the visual neural network model to obtain a first convolution vector set, then performs pooling processing on the first convolution vector set through the pooling layer of the visual neural network model to obtain a first pooling vector set, and finally inputs the first pooling vector set into the first fully connected layer of the visual neural network model through the first channel to obtain a first intermediate analysis result output by the first fully connected layer; the electronic device convolves the second image through the convolution layer of the visual neural network model to obtain a second convolution vector set, then performs pooling processing on the second convolution vector set through the pooling layer of the visual neural network model to obtain a second pooling vector set, and finally inputs the second pooling vector set into the first fully connected layer through the second channel. The electronic device convolves the third image through the convolution layer of the visual neural network model to obtain a third convolution vector set, and then performs pooling processing on the third convolution vector set through the pooling layer of the visual neural network model to obtain a third pooling vector set, and finally inputs the third pooling vector set into the third fully connected layer of the visual neural network model through the third channel to obtain a third intermediate analysis result output by the third fully connected layer; wherein, the first fully connected layer and the second fully connected layer share some neurons, and the second fully connected layer and the third fully connected layer share some neurons to realize association processing in the spatial dimension; the electronic device processes the first intermediate analysis result, the second intermediate analysis result and the third intermediate analysis result through the output layer of the visual neural network model to obtain a processing result output by the output layer.

[0013] Optionally, the neurons included in the first fully connected layer, the neurons included in the second fully connected layer, and the neurons included in the second fully connected layer form a cellular connected network structure, and some of the neurons shared by the first fully connected layer and the second fully connected layer are: neurons in the cellular connected network structure that are directly connected to neurons in the first fully connected layer that are not shared by the second fully connected layer, and are also directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer; some of the neurons shared by the second fully connected layer and the third fully connected layer are: neurons in the cellular connected network structure that are directly connected to neurons in the second fully connected layer that are not shared by the third fully connected layer, and are also directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer; in the cellular connected network structure, neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer, and neurons in the second fully connected layer that are not shared by the third fully connected layer are not directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer.

[0014] Optionally, the visual neural network model is configured as follows: when the visual neural network model processes the first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated; when the visual neural network model processes the second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, neurons in the third fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer and the second fully connected layer are activated; and in the visual neural network When the network model processes the third image, the neurons in the second fully connected layer that are not shared by the third fully connected layer are not activated, and the neurons in the third fully connected layer that are not shared by the second fully connected layer are activated; the first fully connected layer is configured as: a training image set obtained by using the shooting device to shoot the contact points in the first shooting insertion direction is trained until convergence, the second fully connected layer is configured as: a training image set obtained by using the shooting device to shoot the contact points in the second shooting insertion direction is trained until convergence, and the second fully connected layer is configured as: a training image set obtained by using the shooting device to shoot the contact points in the third shooting insertion direction is trained until convergence.

[0015] Optionally, the visual neural network model is an enhanced convolutional neural network model.

[0016] In a second aspect, a contact point detection device for a pin-type electrical connector based on visual nerves is provided, which is applied to electronic equipment and is configured to execute the method described in the first aspect.

[0017] In a third aspect, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are executed on a computer, the computer is caused to execute the method described in the first aspect.

[0018] In summary, the above method and device have the following technical effects:

[0019] For a certain detection, the electronic device can first obtain at least two images of the contact points of the pin electrical connector by photographing the contact points with different shooting directions. Based on the different shooting directions, the electronic device can associate the at least two images in the spatial dimension through the visual neural network model to determine whether there is poor contact at the contact point. At this time, since the output of the result takes into account the characteristics of the spatial dimension, the robustness of the detection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of a flow chart of a contact point detection method of a pin-out electrical connector based on visual nerves provided in an embodiment of the present application;

[0021] Figure 2A schematic diagram of the structure of a pin-out electrical connector in a contact point detection method of a pin-out electrical connector based on visual nerves provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the structure of a network model in a contact point detection method for a pin-to-pin electrical connector based on visual nerves provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of the fully connected layer in the contact point detection method of the pin-out electrical connector based on visual nerves provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0026] The present application will present various aspects, embodiments or features around a system that may include multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in conjunction with the figures. In addition, combinations of these schemes may also be used.

[0027] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0028] In the embodiments of the present application, "information", "signal", "message", "channel" and "signaling" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are matched. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the distinction between them is not emphasized, the meanings to be expressed are matched. In addition, the " / " mentioned in the present application can be used to express an "or" relationship.

[0029] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0030] For example, Figure 1 The present invention provides a schematic flow chart of a method for detecting contact points of a pin-to-pin electrical connector based on visual nerves. The method can be applied to electronic devices.

[0031] like Figure 1 As shown, the process of the contact point detection method of the pin-out electrical connector based on visual nerve is as follows:

[0032] S101, the electronic device acquires at least two images.

[0033] The at least two images are images obtained by photographing the contact points of the pin-to-pin electrical connector from different shooting directions, and each of the at least two images includes the contact points. The photographing device can be a camera or a webcam without specific limitation.

[0034] For example, Figure 2 As shown, the pin power connector 10 includes a base 110 and a pin 120, the pin 120 includes a first end 121 and a second end 121, the first end 121 of the pin 120 is arranged on the base 110, and the contact point for taking power arranged between the first end 121 and the second end 122 includes a first contact point 131 and a second contact point 132. The first contact point 131 and the second contact point are distributed on both sides of the pin 120 (such as symmetrically arranged on both sides), and the shapes of the first contact point 131 and the second contact point 132 are both slightly convex arc structures. A first reference line x1 connecting the first contact point 131 and the second contact point 132 is set, and a second reference line x2 connecting the first end 121 and the second end 122 is set, and the first reference line x1 and the second reference line x2 are located on a reference plane Px, or the first reference line x1 and the second reference line x2 constitute a reference plane Px.

[0035] In the first possible way, Figure 2As shown in (a), at least two images include a first image and a second image. The first image is an image obtained by the shooting device shooting the first contact point 131 and the second contact point 132 in the first shooting insertion direction y1, and the first shooting insertion direction y1 and the reference direction y0 are at a first preset angle. The first preset angle is not 0, such as 3-5°. The second image is an image obtained by the shooting device shooting the first contact point 131 and the second contact point 132 in the second shooting insertion direction y1, and the second preset angle is not 0, such as 3-5°, and the reference direction y0 is a direction perpendicular to the reference plane Px. It can be seen that the first preset angle y1 and the second preset angle y2 are both relatively small angles to ensure that the images of the contact points in the first image and the second image are roughly consistent and the difference is not large. For example, the image of the first contact point 131 in the first image is partially the same as the image of the first contact point 133 in the second image, and the image of the second contact point 132 in the first image is partially the same as the image of the second contact point 132 in the second image, to ensure that the features in the spatial dimension are related.

[0036] In the second possible way, Figure 2 As shown in (b), at least two images include a first image, a second image, and a third image. The first image is an image obtained by photographing the first contact point 131 and the second contact point 132 with a first photographing insertion direction y1 by a photographing device. The first photographing insertion direction is at a first preset angle with the reference direction, and the first preset angle is not 0, such as 3-5°. The second image is an image obtained by photographing the first contact point 131 and the second contact point 132 with a second photographing insertion direction y2 by a photographing device. The second photographing insertion direction y2 is in the same direction as the reference direction y0; the third image is an image obtained by photographing the first contact point 131 and the second contact point 132 with a third photographing insertion direction y3 by a photographing device. The third photographing insertion direction y3 is at a second preset angle with the reference direction y0, and the second preset angle is not 0, such as 3-5°. The reference direction y0 is a direction perpendicular to the reference plane Px. Similarly, the first preset angle y1 and the second preset angle y2 are both relatively small angles, ensuring that the differences between the three shooting directions are not large, so as to ensure that the images of the contact points in the first image, the second image and the third image are roughly consistent, for example, the image of the first contact point 131 in the first image, the image of the first contact point 131 in the second image, and the image of the first contact point 131 in the third image are partially the same; the image of the second contact point 132 in the first image, the image of the second contact point 132 in the second image, and the image of the second contact point 132 in the third image are partially the same, so as to ensure that the features in the spatial dimension are related.

[0037] S102, the electronic device associates at least two images in a spatial dimension through a visual neural network model to obtain a processing result output by the visual neural network model.

[0038] The processing result can indicate whether there is a poor contact situation at the contact point. The visual neural network model is an enhanced convolutional neural network model, such as a convolutional neural network model with enhanced model structure, which is described in detail as follows.

[0039] Continuing with the first possible approach:

[0040] like Figure 3 As shown in (a) in the figure, the visual neural network model includes a convolutional layer, a pooling layer, a first fully connected layer connected to the pooling layer through a first channel, a second fully connected layer connected to the pooling layer through a second channel, and an output layer.

[0041] First, the electronic device convolves the first image through the convolution layer of the visual neural network model (that is, the first image is first input into the visual neural network model, and the convolution layer convolves the first image) to obtain a first set of convolution vectors. The electronic device then performs pooling processing on the first set of convolution vectors through the pooling layer of the visual neural network model to obtain a first set of pooled vectors. Finally, the electronic device inputs the first set of pooled vectors into the first fully connected layer of the visual neural network model through the first channel to obtain a first intermediate analysis result output by the first fully connected layer. At this time, the first intermediate analysis result can be a probability, such as a value between 0 and 1, and the first intermediate analysis result can be output to the output layer cache.

[0042] Afterwards, the electronic device first convolves the second image through the convolution layer of the visual neural network model (at this time, the convolution layer has completed the convolution of the first image) to obtain a second set of convolution vectors. The electronic device then performs pooling processing on the second set of convolution vectors through the pooling layer of the visual neural network model (at this time, the pooling layer has completed the pooling of the first set of convolution vectors) to obtain a second set of pooled vectors. Finally, the electronic device inputs the second set of pooled vectors into the second fully connected layer of the visual neural network model through the second channel to obtain the second intermediate analysis result output by the second fully connected layer. At this time, the second intermediate analysis result can be a probability, such as a value between 0-1, and the second intermediate analysis result can be output to the output layer cache.

[0043] Among them, the first fully connected layer and the second fully connected layer share some neurons to realize the association processing in the spatial dimension. For example, the neurons included in the first fully connected layer and the neurons included in the second fully connected layer form a honeycomb connected network structure, and the neurons shared by the first fully connected layer and the second fully connected layer are: neurons in the honeycomb connected network structure that are directly connected to the neurons in the first fully connected layer that are not shared by the second fully connected layer, and are also directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer. In the honeycomb connected network structure, the neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer. For example, Figure 4 As shown in (a) (where the ellipsis (......) indicates that other neurons are not shown), the unfilled ones are unshared neurons, and the filled ones are shared neurons. The advantage of this structure is that each shared neuron can be connected to the unshared neurons in its own fully connected layer, which can better realize the shared structure and thus achieve better coupling processing in the spatial dimension.

[0044] It can be understood that the visual neural network model can be configured as follows: when the visual neural network model processes the first image, the neurons in the first fully connected layer that are not shared by the second fully connected layer are activated (i.e., in a working state, such as by configuring the loss function of the neuron to be in a valid state, it can be used for calculation, and it can also be reversed during the training process to achieve regression), and the neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated (i.e., not in a working state, such as by configuring the loss function of the neuron to be in an invalid state, it cannot be used for calculation, and thus cannot participate in the processing of the first image, and cannot be reversed during the training process to achieve regression). Similarly, when the visual neural network model processes the second image, the neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, and the neurons in the second fully connected layer that are not shared by the first fully connected layer are activated, and some neurons shared by the first fully connected layer and the second fully connected layer are always activated when the visual neural network model processes the first image and the second image (i.e., the loss function in the shared neurons must participate in the calculation when processing the first image and the second image).

[0045] It can also be understood that the first fully connected layer can be configured as follows: the training image set obtained by using the camera to shoot the contact points in the first shooting direction is trained until convergence (including the shared neurons also participating in the training until convergence), and the second fully connected layer is configured as follows: the training image set obtained by using the camera to shoot the contact points in the second shooting direction is trained until convergence (including the shared neurons also participating in the training until convergence). Specifically, it can also be implemented in the above-mentioned activation and non-activation manner. For example, when training the training image set obtained by using the camera to shoot the contact points in the first shooting direction, the neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and the neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated. The training of the training image set obtained by using the camera to shoot the contact points in the second shooting direction is the same, and will not be repeated. As a result, the shared neurons have the ability to process images of the same object from different perspectives, thereby realizing the coupling of spatial dimensions.

[0046] Finally, the electronic device processes the first intermediate analysis result and the second intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer, such as averaging the value of the first intermediate analysis result and the value of the second intermediate analysis result to obtain the average value. If the average value is greater than a threshold value (such as 0.9), it indicates that there is no poor contact at the contact point. Otherwise, there is poor contact at the contact point.

[0047] It can be understood that the neurons of the present application are prior art, and the difference is that a new connection structure and state (such as activation / inactivation) of the neurons are defined.

[0048] Continuing with the second possible way:

[0049] like Figure 3 As shown in (b), the visual neural network model includes a convolutional layer, a pooling layer, a first fully connected layer connected to the pooling layer through a first channel, a second fully connected layer connected to the pooling layer through a second channel, a third fully connected layer connected to the pooling layer through a third channel, and an output layer.

[0050] First, the electronic device convolves the first image through the convolution layer of the visual neural network model to obtain a first convolution vector set, then performs pooling processing on the first convolution vector set through the pooling layer of the visual neural network model to obtain a first pooling vector set, and finally inputs the first pooling vector set into the first fully connected layer of the visual neural network model through the first channel to obtain a first intermediate analysis result output by the first fully connected layer.

[0051] Then, the electronic device convolves the second image through the convolution layer of the visual neural network model to obtain a second convolution vector set, performs pooling processing on the second convolution vector set through the pooling layer of the visual neural network model to obtain a second pooling vector set, and finally inputs the second pooling vector set into the second fully connected layer of the visual neural network model through the second channel to obtain a second intermediate analysis result output by the second fully connected layer;

[0052] Furthermore, the electronic device convolves the third image through the convolution layer of the visual neural network model to obtain a third convolution vector set, then performs pooling on the third convolution vector set through the pooling layer of the visual neural network model to obtain a third pooled vector set, and finally inputs the third pooled vector set into the third fully connected layer of the visual neural network model through the third channel to obtain a third intermediate analysis result output by the third fully connected layer.

[0053] It can be understood that the above processing method is also processed sequentially, and the logic and principle are similar to the first possible method mentioned above. Please refer to it for understanding and will not repeat it here.

[0054] Among them, the first fully connected layer and the second fully connected layer share some neurons, and the second fully connected layer and the third fully connected layer share some neurons to realize association processing in the spatial dimension.

[0055] For example, the neurons included in the first fully connected layer, the neurons included in the second fully connected layer, and the neurons included in the second fully connected layer form a cellular connected network structure. The neurons shared by the first fully connected layer and the second fully connected layer are: neurons in the cellular connected network structure that are directly connected to the neurons in the first fully connected layer that are not shared by the second fully connected layer, and are also directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer. Similarly, the neurons shared by the second fully connected layer and the third fully connected layer are: neurons in the cellular connected network structure that are directly connected to the neurons in the second fully connected layer that are not shared by the third fully connected layer, and are also directly connected to the neurons in the third fully connected layer that are not shared by the second fully connected layer; in the cellular connected network structure, the neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer, and the neurons in the second fully connected layer that are not shared by the third fully connected layer are not directly connected to the neurons in the third fully connected layer that are not shared by the second fully connected layer. For example, Figure 4As shown in (b) (where the ellipsis (......) indicates that other neurons are not shown), the unfilled pattern 1 is part of the neurons shared by the first fully connected layer and the second fully connected layer, the unfilled pattern 2 is part of the neurons shared by the second fully connected layer and the third fully connected layer, and the filled ones are the shared neurons. Similarly, the advantage of this structure is that each shared neuron can be connected to the unshared neurons in its own fully connected layer, which can better realize the shared structure and thus better realize the coupling processing of the spatial dimension.

[0056] It can be understood that the visual neural network model can also be configured as follows: when the visual neural network model processes the first image, the neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and the neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated; when the visual neural network model processes the second image, the neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, the neurons in the third fully connected layer that are not shared by the second fully connected layer are not activated, and the neurons in the second fully connected layer that are not shared by the first and second fully connected layers are activated; and when the visual neural network model processes the third image, the neurons in the second fully connected layer that are not shared by the third fully connected layer are not activated, and the neurons in the third fully connected layer that are not shared by the second fully connected layer are activated. The specific principle is similar to the first possible method mentioned above. Please refer to it for understanding and will not be repeated here.

[0057] Similarly, the first fully connected layer is further configured to train a set of training images obtained by using the photographing device to photograph the contact points in a first photographing insertion direction until convergence, the second fully connected layer is further configured to train a set of training images obtained by using the photographing device to photograph the contact points in a second photographing insertion direction until convergence, and the second fully connected layer is further configured to train a set of training images obtained by using the photographing device to photograph the contact points in a third photographing insertion direction until convergence. As a result, the shared neurons have the ability to process images of the same object from two different perspectives, thereby realizing coupling of spatial dimensions. The specific principle is similar to the first possible method mentioned above. Please refer to it for understanding and will not be repeated here.

[0058] The electronic device processes the first intermediate analysis result, the second intermediate analysis result and the third intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer, such as averaging the values ​​of the first intermediate analysis result, the second intermediate analysis result and the third intermediate analysis result to obtain an average value. If the average value is greater than a threshold value (such as 0.9), it indicates that there is no poor contact at the contact point; otherwise, there is poor contact at the contact point.

[0059] In summary: for a certain detection, the electronic device can first obtain at least two images of the contact points of the pin electrical connector by photographing the contact points with different shooting directions. Based on the different shooting directions, the electronic device can associate the at least two images in the spatial dimension through the visual neural network model to determine whether there is poor contact at the contact point. At this time, since the output of the result takes into account the characteristics of the spatial dimension, the robustness of the detection can be improved.

[0060] Combination of the above Figure 1 The contact point detection method of the pin-to-pin electrical connector based on visual nerves provided in the embodiment of the present application is described in detail. The following is a detailed description of a contact point detection device of the pin-to-pin electrical connector based on visual nerves for executing the contact point detection method of the pin-to-pin electrical connector based on visual nerves provided in the embodiment of the present application. The device is applied to electronic equipment and is configured to execute the above Figure 1 The method shown.

[0061] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. For example, the electronic device may be a terminal device, or a chip (system) or other component or assembly that can be provided in the terminal device. Figure 5 As shown, the electronic device 400 may include a processor 401. Optionally, the electronic device 400 may also include a memory 402 and / or a transceiver 403. The processor 401 is coupled to the memory 402 and the transceiver 403, such as being connected via a communication bus. In addition, the electronic device 400 may also be a chip, such as including the processor 401, in which case the transceiver may be an output and input interface of the chip.

[0062] Combine the following Figure 5 The components of the electronic device 400 are specifically introduced as follows:

[0063] The processor 401 is the control center of the electronic device 400, and may be a processor or a general term for multiple processing elements. For example, the processor 401 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be configured to implement one or more integrated circuits of the embodiments of the present application, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0064] Optionally, the processor 401 can execute various functions of the electronic device 400 by running or executing the software program stored in the memory 402 and calling the scientific data stored in the memory 402, such as executing the above Figure 1 The contact point detection method of the pin-out electrical connector based on visual nerve is shown.

[0065] In a specific implementation, as an embodiment, the processor 401 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 are shown in FIG.

[0066] In a specific implementation, as an embodiment, the electronic device 400 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing scientific data (such as computer programs or instructions).

[0067] The memory 402 is used to store the software program for executing the solution of the present application, and the execution is controlled by the processor 401. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0068] Optionally, the memory 402 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or scientific data structures and can be accessed by a computer, but is not limited thereto. The memory 402 may be integrated with the processor 401, or may exist independently and accessed through the interface circuit ( Figure 5 (not shown) is coupled to the processor 401, which is not specifically limited in the embodiment of the present application.

[0069] The transceiver 403 is used for communication with other electronic devices. For example, if the electronic device 400 is a terminal device, the transceiver 403 can be used to communicate with a network device, or with another terminal device. For another example, if the electronic device 400 is a network device, the transceiver 403 can be used to communicate with a terminal device, or with another network device.

[0070] Optionally, the transceiver 403 may include a receiver and a transmitter ( Figure 5 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0071] Optionally, the transceiver 403 may be integrated with the processor 401, or may exist independently and communicate with the electronic device 400 through an interface circuit ( Figure 5 (not shown) is coupled to the processor 401, which is not specifically limited in the embodiment of the present application.

[0072] Understandably, Figure 5 The structure of the electronic device 400 shown in the figure does not constitute a limitation on the electronic device, and the actual electronic device may include more or fewer components than those shown in the figure, or combine certain components, or arrange the components differently.

[0073] In addition, the technical effects of the electronic device 400 can refer to the technical effects of the methods described in the above method embodiments, which will not be repeated here.

[0074] It should be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0075] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0076] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or scientific data center to another website, computer, server or scientific data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a scientific data storage device such as a server, scientific data center, etc. that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0077] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0078] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0079] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0080] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0081] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0082] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0084] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0085] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0086] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A contact point detection method for a pin-out electrical connector based on visual nerves, characterized in that: Applied to electronic equipment, the method comprises: The electronic device acquires at least two images, wherein the at least two images are images obtained by photographing the contact points of the pin-to-pin electrical connector with different photographing directions by a photographing device, and each of the at least two images includes the contact points; The electronic device associates and processes the at least two images in a spatial dimension through a visual neural network model to obtain a processing result output by the visual neural network model, and the processing result indicates whether there is a poor contact situation at the contact point.

2. The method according to claim 1, characterized in that The pin power connector includes a base and a pin, the pin includes a first end and a second end, the first end of the pin is arranged on the base, the contact point for taking power arranged between the first end and the second end includes a first contact point and a second contact point, the first contact point and the second contact point are distributed on both sides of the pin, a first reference line connecting the first contact point and the second contact point is arranged, and a second reference line connecting the first end and the second end is arranged, and the first reference line and the second reference line are located in a reference plane; The at least two images include a first image and a second image, the first image is an image obtained by the photographing device photographing the first contact point and the second contact point in a first photographing insertion direction, the first photographing insertion direction is at a first preset angle to a reference direction, and the first preset angle is not 0, the second image is an image obtained by the photographing device photographing the first contact point and the second contact point in a second photographing insertion direction, the second photographing insertion direction is at a second preset angle to the reference direction, and the second preset angle is not 0, and the reference direction is a direction perpendicular to the reference plane; The image of the first contact point in the first image is partially identical to the image of the first contact point in the second image; The image of the second contact point in the first image is partially identical to the image of the second contact point in the second image.

3. The method according to claim 2, characterized in that The electronic device associates the at least two images in a spatial dimension through a visual neural network model to obtain a processing result output by the visual neural network model, including: The electronic device convolves the first image through the convolution layer of the visual neural network model to obtain a first convolution vector set, then performs pooling processing on the first convolution vector set through the pooling layer of the visual neural network model to obtain a first pooling vector set, and finally inputs the first pooling vector set into the first fully connected layer of the visual neural network model through the first channel to obtain a first intermediate analysis result output by the first fully connected layer; The electronic device convolves the second image through the convolution layer of the visual neural network model to obtain a second convolution vector set, then performs pooling processing on the second convolution vector set through the pooling layer of the visual neural network model to obtain a second pooling vector set, and finally inputs the second pooling vector set into the second fully connected layer of the visual neural network model through a second channel to obtain a second intermediate analysis result output by the second fully connected layer; Wherein, the first fully connected layer and the second fully connected layer share some neurons to implement the association processing in the spatial dimension; The electronic device processes the first intermediate analysis result and the second intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer.

4. The method according to claim 3, characterized in that The neurons included in the first fully connected layer and the neurons included in the second fully connected layer form a honeycomb connected network structure, and the part of neurons shared by the first fully connected layer and the second fully connected layer are: neurons in the honeycomb connected network structure that are directly connected to the neurons in the first fully connected layer that are not shared by the second fully connected layer, and are also directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer; in the honeycomb connected network structure, the neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer.

5. The method according to claim 4, characterized in that The visual neural network model is configured such that: when the visual neural network model processes the first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated, and when the visual neural network model processes the second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are activated, and the part of neurons shared by the first fully connected layer and the second fully connected layer are always activated when the visual neural network model processes the first image and the second image; The first fully connected layer is configured to train a training image set obtained by photographing the contact points with the first photographing insertion direction using the photographing device until convergence, and the second fully connected layer is configured to train a training image set obtained by photographing the contact points with the second photographing insertion direction using the photographing device until convergence.

6. The method according to claim 1, characterized in that The pin power connector includes a base and a pin, the pin includes a first end and a second end, the first end of the pin is arranged on the base, the contact point for taking power arranged between the first end and the second end includes a first contact point and a second contact point, the first contact point and the second contact point are distributed on both sides of the pin, a first reference line connecting the first contact point and the second contact point is arranged, and a second reference line connecting the first end and the second end is arranged, and the first reference line and the second reference line are located in a reference plane; The at least two images include a first image, a second image, and a third image, wherein the first image is an image obtained by the photographing device photographing the first contact point and the second contact point in a first photographing direction, the first photographing direction and a reference direction are at a first preset angle, and the first preset angle is not 0; the second image is an image obtained by the photographing device photographing the first contact point and the second contact point in a second photographing direction, and the second photographing direction and the reference direction are in the same direction; the third image is an image obtained by the photographing device photographing the first contact point and the second contact point in a third photographing direction, the third photographing direction and the reference direction are at a second preset angle, and the second preset angle is not 0, and the reference direction is a direction perpendicular to the reference plane; The image of the first contact point in the first image, the image of the first contact point in the second image, and the image of the first contact point in the third image are partially identical; The image of the second contact point in the first image, the image of the second contact point in the second image, and the image of the second contact point in the third image are partially identical.

7. The method according to claim 6, characterized in that The electronic device associates the at least two images in a spatial dimension through a visual neural network model to obtain a processing result output by the visual neural network model, including: The electronic device convolves the first image through the convolution layer of the visual neural network model to obtain a first convolution vector set, then performs pooling processing on the first convolution vector set through the pooling layer of the visual neural network model to obtain a first pooling vector set, and finally inputs the first pooling vector set into the first fully connected layer of the visual neural network model through the first channel to obtain a first intermediate analysis result output by the first fully connected layer; The electronic device convolves the second image through the convolution layer of the visual neural network model to obtain a second convolution vector set, then performs pooling processing on the second convolution vector set through the pooling layer of the visual neural network model to obtain a second pooling vector set, and finally inputs the second pooling vector set into the second fully connected layer of the visual neural network model through a second channel to obtain a second intermediate analysis result output by the second fully connected layer; The electronic device convolves the third image through the convolution layer of the visual neural network model to obtain a third convolution vector set, then performs pooling processing on the third convolution vector set through the pooling layer of the visual neural network model to obtain a third pooling vector set, and finally inputs the third pooling vector set into the third fully connected layer of the visual neural network model through a third channel to obtain a third intermediate analysis result output by the third fully connected layer; The first fully connected layer and the second fully connected layer share some neurons, and the second fully connected layer and the third fully connected layer share some neurons to implement the association processing in the spatial dimension; The electronic device processes the first intermediate analysis result, the second intermediate analysis result and the third intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer.

8. The method according to claim 7, characterized in that The neurons included in the first fully connected layer, the neurons included in the second fully connected layer and the neurons included in the second fully connected layer form a cellular connected network structure, and the part of neurons shared by the first fully connected layer and the second fully connected layer are: neurons in the cellular connected network structure that are directly connected to the neurons in the first fully connected layer that are not shared by the second fully connected layer, and are also directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer; the part of neurons shared by the second fully connected layer and the third fully connected layer are: neurons in the cellular connected network structure that are directly connected to the neurons in the second fully connected layer that are not shared by the third fully connected layer, and are also directly connected to the neurons in the third fully connected layer that are not shared by the second fully connected layer; in the cellular connected network structure, the neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to the neurons in the second fully connected layer that are not shared by the first fully connected layer, and the neurons in the second fully connected layer that are not shared by the third fully connected layer are not directly connected to the neurons in the third fully connected layer that are not shared by the second fully connected layer.

9. The method according to claim 8, characterized in that The visual neural network model is configured as follows: when the visual neural network model processes the first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated; when the visual neural network model processes the second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, neurons in the third fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer and the second fully connected layer are activated; and when the visual neural network model processes the third image, neurons in the second fully connected layer that are not shared by the third fully connected layer are not activated, and neurons in the third fully connected layer that are not shared by the second fully connected layer are activated; The first fully connected layer is configured to: use the shooting device to shoot the contact points with the first shooting insertion direction to obtain a training image set for training until convergence, the second fully connected layer is configured to: use the shooting device to shoot the contact points with the second shooting insertion direction to obtain a training image set for training until convergence, and the second fully connected layer is configured to: use the shooting device to shoot the contact points with the third shooting insertion direction to obtain a training image set for training until convergence.

10. A contact point detection device for a pin-out electrical connector based on visual nerves, characterized in that: Applied to an electronic device, the device is configured to execute the method as claimed in any one of claims 1 to 9.

Images