A contact point detection method and device for a pin power taking connector based on a visual nerve

Abstract

The application provides a contact point detection method and device for a pin power taking connector based on a visual nerve, belongs to the technical field of image processing, and guarantees the robustness of detection. The method is applied to an electronic device and comprises the following steps: the electronic device acquires at least two images, the at least two images are images obtained by shooting a contact point of a pin power taking connector by a shooting device in different shooting directions, and each of the at least two images contains the contact point; and the electronic device processes the at least two images in a spatial dimension by a visual nerve network model to obtain a processing result output by the visual nerve network model, and the processing result indicates whether the contact point has a poor contact condition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for detecting contact points of a pin-type power connector based on visual nerves. Background Technology

[0002] With the deepening development of industrial automation and intelligent manufacturing, the performance and reliability of electrical equipment have become crucial factors affecting production efficiency and safety. As a key component of power-generating equipment, the quality of the contact points of pin-type power connectors directly impacts the stability of the equipment. Currently, traditional contact point detection methods mainly rely on manual experience and equipment performance testing. However, manual inspection is time-consuming and difficult to meet the demands of large-scale production lines. Furthermore, manual inspection, dependent on the experience of inspectors, is prone to misjudgments. Therefore, a neural network model is considered. For example, the collected data is first preprocessed to extract key features, preparing for the training of the neural network model. Then, a suitable neural network architecture is selected, and the model is trained and optimized to improve the accuracy and robustness of the detection. Finally, the trained model is deployed in the actual production environment to achieve automated detection, and the model's performance is continuously monitored to ensure the reliability of the detection results.

[0003] However, ensuring the robustness of neural network models remains a research challenge. Summary of the Invention

[0004] This application provides a method and apparatus for detecting contact points of a pin-type power connector based on visual nerves, in order to ensure the robustness of the detection.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, a method for detecting contact points of a pin-feed connector based on visual neural networks is provided, which is applied to electronic devices. The method includes: the electronic device acquiring at least two images, which are images of the contact points of the pin-feed connector captured by an imaging device from different shooting directions, and each of the at least two images contains a contact point; the electronic device processing the at least two images in spatial dimension through a visual neural network model to obtain the processing result output by the visual neural network model, and the processing result indicating whether there is a poor contact at the contact point.

[0007] Optionally, the pin-type power connector includes a base and pins. Each pin includes a first end and a second end. The first end of the pin is disposed on the base. A power-generating contact point is provided between the first and second ends, including a first contact point and a second contact point. The first and second contact points are distributed on both sides of the pin. A first reference line is provided connecting the first and second contact points, and a second reference line is provided between the first and second ends. The first and second reference lines are located on a reference plane. At least two images are included, including a first image and a second image. The first image is obtained by a camera capturing the first and second contact points from a first shooting direction. The first shooting direction and the reference direction are at a first preset angle, which is not zero. The second image is obtained by a camera capturing the first and second contact points from a second shooting direction. The second shooting direction and the reference direction are at a second preset angle, which is not zero. The reference direction is perpendicular to the reference plane. The image of the first contact point in the first image is partially identical to the image of the first contact point in the second image. The image of the second contact point in the first image is partially identical to the image of the second contact point in the second image.

[0008] Optionally, the electronic device uses a visual neural network model to correlate at least two images in a spatial dimension to obtain the processing result output by the visual neural network model. This includes: the electronic device convolves the first image using a convolutional layer of the visual neural network model to obtain a first set of convolutional vectors; then pools the first set of convolutional vectors using a pooling layer of the visual neural network model to obtain a first set of pooled vectors; finally, the first set of pooled vectors is input into the first fully connected layer of the visual neural network model through a first channel to obtain a first intermediate analysis result output by the first fully connected layer; the electronic device convolves the second image using a convolutional layer of the visual neural network model to obtain a second set of convolutional vectors; then pools the second set of convolutional vectors using a pooling layer of the visual neural network model to obtain a second set of pooled vectors; finally, the second set of pooled vectors is input into the second fully connected layer of the visual neural network model through a second channel to obtain a second intermediate analysis result output by the second fully connected layer; wherein the first and second fully connected layers share some neurons for spatial correlation processing; the electronic device processes the first and second intermediate analysis results through the output layer of the visual neural network model to obtain the processing result output by the output layer.

[0009] Optionally, the neurons in the first fully connected layer and the neurons in the second fully connected layer form a cellular network structure. The neurons shared by the first and second fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the first fully connected layer that are not shared by the second fully connected layer, and also directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer. In the cellular network structure, neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer.

[0010] Optionally, the visual neural network model is configured such that: when the visual neural network model processes a first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated; and when the visual neural network model processes a second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are activated; and some neurons shared by the first and second fully connected layers are always activated when the visual neural network model processes the first and second images; the first fully connected layer is configured to be trained to convergence using a set of training images obtained by shooting the contact point with the shooting device in a first shooting direction; and the second fully connected layer is configured to be trained to convergence using a set of training images obtained by shooting the contact point with the shooting device in a second shooting direction.

[0011] Optionally, the pin-type power connector includes a base and pins. Each pin includes a first end and a second end. The first end of the pin is disposed on the base. A power-generating contact point is provided between the first and second ends, including a first contact point and a second contact point. The first and second contact points are distributed on both sides of the pin. A first reference line is provided connecting the first and second contact points, and a second reference line is provided between the first and second ends. The first and second reference lines are located on a reference plane. At least two images are included, including a first image, a second image, and a third image. The first image is an image obtained by the imaging device capturing the first and second contact points from a first shooting direction. The first shooting direction and the reference direction are at a first preset angle. The preset angle is not 0. The second image is an image obtained by the shooting device shooting the first contact point and the second contact point in a second shooting direction, and the second shooting direction is the same as the reference direction. The third image is an image obtained by the shooting device shooting the first contact point and the second contact point in a third shooting direction, and the third shooting direction is at a second preset angle with the reference direction. The second preset angle is not 0, and the reference direction is a direction perpendicular to the reference plane. The images of the first contact point in the first image, the first contact point in the second image, and the first contact point in the third image are partially the same. The images of the second contact point in the first image, the second contact point in the second image, and the second contact point in the third image are partially the same.

[0012] Optionally, the electronic device performs spatial correlation processing on at least two images using a visual neural network model to obtain the processing result output by the visual neural network model. This includes: the electronic device convolves the first image using a convolutional layer of the visual neural network model to obtain a first set of convolutional vectors; then pools the first set of convolutional vectors using a pooling layer of the visual neural network model to obtain a first set of pooled vectors; finally, the first set of pooled vectors is input into the first fully connected layer of the visual neural network model through a first channel to obtain a first intermediate analysis result output by the first fully connected layer; the electronic device convolves the second image using a convolutional layer of the visual neural network model to obtain a second set of convolutional vectors; then pools the second set of convolutional vectors using a pooling layer of the visual neural network model to obtain a second set of pooled vectors; finally, the second set of pooled vectors is input into the second channel. The second fully connected layer of the visual neural network model outputs a second intermediate analysis result. The electronic device convolves the third image through the convolutional layer of the visual neural network model to obtain a third set of convolutional vectors. Then, the third set of convolutional vectors is pooled through the pooling layer of the visual neural network model to obtain a third set of pooled vectors. Finally, the third set of pooled vectors is input into the third fully connected layer of the visual neural network model through the third channel to obtain the third intermediate analysis result output by the third fully connected layer. The first and second fully connected layers share some neurons, and the second and third fully connected layers share some neurons to achieve spatial correlation processing. The electronic device processes the first, second, and third intermediate analysis results through the output layer of the visual neural network model to obtain the processing result output by the output layer.

[0013] Optionally, the neurons included in the first fully connected layer, the neurons included in the second fully connected layer, and the neurons included in the third fully connected layer form a cellular network structure. The neurons shared by the first and second fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the first fully connected layer that are not shared by the second fully connected layer, and also directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer. The neurons shared by the second and third fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the second fully connected layer that are not shared by the third fully connected layer, and also directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer. In the cellular network structure, neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer, and neurons in the second fully connected layer that are not shared by the third fully connected layer are not directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer.

[0014] Optionally, the visual neural network model is configured such that: when the visual neural network model processes a first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated; when the visual neural network model processes a second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, neurons in the third fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first and second fully connected layers are activated; and in the visual neural network model processing a second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the third fully connected layer that are not shared by the second fully connected layer are activated. When the network model processes the third image, neurons in the second fully connected layer that are not shared by the third fully connected layer are not activated, while neurons in the third fully connected layer that are not shared by the second fully connected layer are activated. The first fully connected layer is configured to train until convergence using a set of training images obtained by photographing the contact point with the camera in the first shooting direction. The second fully connected layer is configured to train until convergence using a set of training images obtained by photographing the contact point with the camera in the second shooting direction. The second fully connected layer is configured to train until convergence using a set of training images obtained by photographing the contact point with the camera in the third shooting direction.

[0015] Optionally, the visual neural network model is an enhanced convolutional neural network model.

[0016] In a second aspect, a contact point detection device for a optic nerve-based pin-feed connector is provided for use in an electronic device, the device being configured to perform the method described in the first aspect.

[0017] Thirdly, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are executed on a computer, the computer causes the computer to perform the method described in the first aspect.

[0018] In summary, the above method and apparatus have the following technical effects:

[0019] For a given detection, the electronic device can first acquire at least two images of the contact point of the pin-feed connector taken by the camera from different shooting directions. Based on the different shooting directions, the electronic device can correlate the at least two images in the spatial dimension through a visual neural network model to determine whether there is a poor contact at the contact point. Since the output of the result takes into account the features of the spatial dimension, the robustness of the detection can be improved. Attached Figure Description

[0020] Figure 1 A schematic flowchart of the contact point detection method for a pin-type power connector based on visual nerves provided in the embodiments of this application;

[0021] Figure 2A schematic diagram of the pin-feed connector structure in the contact point detection method for the pin-feed connector based on visual nerve provided in the embodiments of this application;

[0022] Figure 3 A schematic diagram of the network model in the contact point detection method for a pin-type power connector based on visual neural network provided in the embodiments of this application;

[0023] Figure 4 A schematic diagram of the structure of the fully connected layer in the contact point detection method for a pin-type power connector based on visual nerves provided in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0026] This application will present various aspects, embodiments, or features relating to systems that may include multiple devices, components, modules, etc. It should be understood and appreciated that individual systems may include additional devices, components, modules, etc., and / or may not include all the devices, components, modules, etc. discussed in conjunction with the accompanying drawings. Furthermore, combinations of these approaches are also possible.

[0027] Furthermore, in the embodiments of this application, the words "exemplary," "for example," etc., are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the term "exemplary" is intended to present the concept in a concrete manner.

[0028] In the embodiments of this application, the terms "information," "signal," "message," "channel," and "singaling" may sometimes be used interchangeably. It should be noted that, without emphasizing their distinction, their intended meanings are consistent. Similarly, "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing their distinction, their intended meanings are consistent. Furthermore, the " / " mentioned in this application can be used to indicate an "or" relationship.

[0029] The network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0030] For example, Figure 1 This application provides a schematic flowchart of a contact point detection method for a pin-type power connector based on visual nerve perception. This method can be applied to electronic devices.

[0031] like Figure 1 As shown, the flowchart of the contact point detection method for the pin-feed connector based on visual neural networks is as follows:

[0032] S101, the electronic device acquires at least two images.

[0033] At least two images are taken by an imaging device from different shooting directions of the pin-feed connector contact points, and each of the at least two images contains the contact points. The imaging device can be a camera or a webcam, with no specific limitation.

[0034] For example, such as Figure 2 As shown, the pin-type power connector 10 includes a base 110 and pins 120. Pins 120 include a first end 121 and a second end 122. The first end 121 of pin 120 is disposed on the base 110. Contact points for power extraction, including a first contact point 131 and a second contact point 132, are provided between the first end 121 and the second end 122. The first contact point 131 and the second contact point 132 are distributed on both sides of the pin 120 (e.g., symmetrically arranged on both sides). Both the first contact point 131 and the second contact point 132 have slightly convex arc-shaped structures. A first reference line x1 is provided to connect the first contact point 131 and the second contact point 132, and a second reference line x2 is provided to connect the first end 121 and the second end 122. The first reference line x1 and the second reference line x2 are located on a reference plane Px, or in other words, the first reference line x1 and the second reference line x2 constitute the reference plane Px.

[0035] In the first possible approach, such as Figure 2As shown in (a), at least two images include a first image and a second image. The first image is obtained by the imaging device capturing images of the first contact point 131 and the second contact point 132 in a first shooting direction y1. The first shooting direction y1 and the reference direction y0 are at a first preset angle. The first preset angle is not 0, such as 3-5°. The second image is obtained by the imaging device capturing images of the first contact point 131 and the second contact point 132 in a second shooting direction y1. The second shooting direction y2 and the reference direction y0 are at a second preset angle. The second preset angle is not 0, such as 3-5°. The reference direction y0 is a direction perpendicular to the reference plane Px. It can be seen that the first preset angle y1 and the second preset angle y2 are both relatively small angles to ensure that the images of the contact points in the first image and the second image are roughly consistent with each other, with little difference. For example, the image of the first contact point 131 in the first image is partially the same as the image of the first contact point 133 in the second image, and the image of the second contact point 132 in the first image is partially the same as the image of the second contact point 132 in the second image, to ensure that the features in the spatial dimension are related.

[0036] In the second possible approach, such as Figure 2 As shown in (b), at least two images include a first image, a second image, and a third image. The first image is obtained by the imaging device capturing images of the first contact point 131 and the second contact point 132 in a first shooting direction y1. The first shooting direction and the reference direction are at a first preset angle, which is not 0, such as 3-5°. The second image is obtained by the imaging device capturing images of the first contact point 131 and the second contact point 132 in a second shooting direction y2. The second shooting direction y2 is the same as the reference direction y0. The third image is obtained by the imaging device capturing images of the first contact point 131 and the second contact point 132 in a third shooting direction y3. The third shooting direction y3 and the reference direction y0 are at a second preset angle, which is not 0, such as 3-5°. The reference direction y0 is a direction perpendicular to the reference plane Px. Similarly, the first preset angle y1 and the second preset angle y2 are both relatively small angles to ensure that the differences between the three shooting directions are not significant, so as to ensure that the images of the contact points in the first image, the second image and the third image are roughly consistent. For example, the images of the first contact point 131 in the first image, the first contact point 131 in the second image and the first contact point 131 in the third image are partially the same; the images of the second contact point 132 in the first image, the second contact point 132 in the second image and the second contact point 132 in the third image are partially the same, so as to ensure that the features in the spatial dimension are related.

[0037] S102, the electronic device uses a visual neural network model to correlate and process at least two images in a spatial dimension, and obtains the processing result output by the visual neural network model.

[0038] The processing results can indicate whether there is poor contact at the contact point. The visual neural network model is an enhanced convolutional neural network model, such as a model structure-enhanced convolutional neural network model, as described below.

[0039] Continuing with the first possible approach:

[0040] like Figure 3 As shown in (a), the visual neural network model sequentially includes a convolutional layer, a pooling layer, a first fully connected layer connected to the pooling layer through a first channel, a second fully connected layer connected to the pooling layer through a second channel, and an output layer.

[0041] First, the electronic device convolves the first image through the convolutional layer of the visual neural network model (i.e., the first image is first input into the visual neural network model and then convolved by the convolutional layer), obtaining a first set of convolutional vectors. The electronic device then pools the first set of convolutional vectors through the pooling layer of the visual neural network model, obtaining a first set of pooled vectors. Finally, the electronic device inputs the first set of pooled vectors into the first fully connected layer of the visual neural network model through the first channel, obtaining the first intermediate analysis result output by the first fully connected layer. At this point, the first intermediate analysis result can be a probability, such as a value between 0 and 1, and this first intermediate analysis result can be output to the output layer buffer.

[0042] Next, the electronic device first convolves the second image through the convolutional layer of the visual neural network model (at this point, the convolution of the first image by the convolutional layer has already been completed), obtaining a second set of convolutional vectors. Then, the electronic device performs pooling processing on the second set of convolutional vectors through the pooling layer of the visual neural network model (at this point, the pooling of the first set of convolutional vectors by the pooling layer has already been completed), obtaining a second set of pooled vectors. Finally, the electronic device inputs the second set of pooled vectors into the second fully connected layer of the visual neural network model through the second channel, obtaining the second intermediate analysis result output by the second fully connected layer. This second intermediate analysis result can be a probability, such as a value between 0 and 1, and can be output to the output layer buffer.

[0043] In this network, the first fully connected layer and the second fully connected layer share some neurons to achieve spatial association processing. For example, the neurons in the first fully connected layer and the neurons in the second fully connected layer form a cellular network structure. The shared neurons in the first and second fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the first fully connected layer that are not shared by the second fully connected layer, and also directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer. In the cellular network structure, neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer. For example, Figure 4 As shown in (a) (where the ellipsis (...) indicates that other neurons are not shown), the unfilled neurons are the unshared neurons, and the filled neurons are the shared neurons. The advantage of this structure is that each shared neuron can be connected to the unshared neurons in its own fully connected layer, which can better realize the shared structure and thus achieve better coupling processing of spatial dimensions.

[0044] It can be understood that the visual neural network model can be configured such that, when the visual neural network model processes the first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated (i.e., in a working state; for example, by configuring the loss function of the neuron to be in an effective state, it can be used for computation and can also be backpropagated during the training process to achieve regression), and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated (i.e., not in a working state; for example, by configuring the loss function of the neuron to be in an ineffective state, it cannot be used for computation, and therefore cannot participate in the processing of the first image, nor can it be backpropagated during the training process to achieve regression). Similarly, when the visual neural network model processes the second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are activated, and the neurons shared by the first and second fully connected layers are always activated when the visual neural network model processes the first and second images (i.e., the loss function within the shared neurons participates in the computation when processing the first and second images).

[0045] It can also be understood that the first fully connected layer can be configured to train on a set of training images obtained by photographing the contact point from a first shooting direction using an imaging device until convergence (including the shared neurons participating in the training until convergence). The second fully connected layer can be configured to train on a set of training images obtained by photographing the contact point from a second shooting direction using an imaging device until convergence (including the shared neurons participating in the training until convergence). Specifically, this can also be achieved using the activation and deactivation methods described above. For example, when training with a set of training images obtained by photographing the contact point from a first shooting direction using an imaging device, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated. The same principle applies when training with a set of training images obtained by photographing the contact point from a second shooting direction using an imaging device, which will not be elaborated further. Thus, the shared neurons have the ability to process images of the same object from different perspectives, achieving spatial dimensional coupling.

[0046] Finally, the electronic device processes the first and second intermediate analysis results through the output layer of the visual neural network model to obtain the processing result output by the output layer. For example, the average value of the first and second intermediate analysis results is calculated. If the average value is greater than a threshold (e.g., 0.9), it indicates that there is no poor contact at the contact point; otherwise, there is poor contact at the contact point.

[0047] It is understood that the neurons in this application are existing technologies, the difference being that a new connection structure and state (such as activation / deactivation) of the neurons are defined.

[0048] Continuing with the second possible approach:

[0049] like Figure 3 As shown in (b), the visual neural network model sequentially includes a convolutional layer, a pooling layer, a first fully connected layer connected to the pooling layer through a first channel, a second fully connected layer connected to the pooling layer through a second channel, a third fully connected layer connected to the pooling layer through a third channel, and an output layer.

[0050] First, the electronic device convolves the first image through the convolutional layer of the visual neural network model to obtain the first set of convolutional vectors. Then, it performs pooling processing on the first set of convolutional vectors through the pooling layer of the visual neural network model to obtain the first set of pooled vectors. Finally, the first set of pooled vectors is input into the first fully connected layer of the visual neural network model through the first channel to obtain the first intermediate analysis result output by the first fully connected layer.

[0051] Then, the electronic device convolves the second image through the convolutional layer of the visual neural network model to obtain the second convolutional vector set. Then, it performs pooling processing on the second convolutional vector set through the pooling layer of the visual neural network model to obtain the second pooling vector set. Finally, the second pooling vector set is input into the second fully connected layer of the visual neural network model through the second channel to obtain the second intermediate analysis result output by the second fully connected layer.

[0052] Furthermore, the electronic device convolves the third image through the convolutional layer of the visual neural network model to obtain a third set of convolutional vectors. Then, it performs pooling processing on the third set of convolutional vectors through the pooling layer of the visual neural network model to obtain a third set of pooled vectors. Finally, the third set of pooled vectors is input into the third fully connected layer of the visual neural network model through the third channel to obtain the third intermediate analysis result output by the third fully connected layer.

[0053] It is understandable that the above processing method is also sequential, and the logic and principle are similar to the first possible method mentioned above. Please refer to it for understanding, and it will not be repeated here.

[0054] The first fully connected layer shares some neurons with the second fully connected layer, and the second fully connected layer shares some neurons with the third fully connected layer to achieve spatial correlation processing.

[0055] For example, the neurons in the first fully connected layer, the neurons in the second fully connected layer, and the neurons in the third fully connected layer form a cellular network structure. The neurons shared between the first and second fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the first fully connected layer that are not shared by the second fully connected layer, and also directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer. Similarly, the neurons shared between the second and third fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the second fully connected layer that are not shared by the third fully connected layer, and also directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer. In the cellular network structure, neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer, and neurons in the second fully connected layer that are not shared by the third fully connected layer are not directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer. For example, Figure 4As shown in (b) (where ellipses (...) indicate that other neurons are not shown), the unfilled pattern 1 represents neurons shared by the first and second fully connected layers, the unfilled pattern 2 represents neurons shared by the second and third fully connected layers, and the filled patterns represent shared neurons. Similarly, the advantage of this structure is that each shared neuron can connect to the unshared neurons in its respective fully connected layer, which can better realize the shared structure and thus achieve better spatial coupling processing.

[0056] It is understandable that the visual neural network model can also be configured such that, when the visual neural network model processes the first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated; when the visual neural network model processes the second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, neurons in the third fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first and second fully connected layers are activated; and when the visual neural network model processes the third image, neurons in the second fully connected layer that are not shared by the third fully connected layer are not activated, and neurons in the third fully connected layer that are not shared by the second fully connected layer are activated. The specific principle is similar to the first possible method mentioned above, please refer to the explanation, and will not be repeated here.

[0057] Similarly, the first fully connected layer is also configured to train until convergence using a set of training images obtained by shooting the contact point with the camera in the first shooting direction. The second fully connected layer is also configured to train until convergence using a set of training images obtained by shooting the contact point with the camera in the second shooting direction. The second fully connected layer is also configured to train until convergence using a set of training images obtained by shooting the contact point with the camera in the third shooting direction. Thus, the shared neurons have the ability to process images of the same object from two different perspectives, achieving spatial dimension coupling. The specific principle is similar to the first possible method mentioned above. Please refer to it for understanding, and it will not be elaborated here.

[0058] The electronic device processes the first intermediate analysis result, the second intermediate analysis result, and the third intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer. For example, the average value of the first intermediate analysis result, the second intermediate analysis result, and the third intermediate analysis result is calculated. If the average value is greater than a threshold (such as 0.9), it indicates that there is no poor contact at the contact point; otherwise, there is poor contact at the contact point.

[0059] In summary: For a given detection, the electronic device can first acquire at least two images of the contact point of the pin-feed connector taken by the camera from different shooting directions. Based on the different shooting directions, the electronic device can correlate and process the at least two images in the spatial dimension through a visual neural network model to determine whether there is a poor contact at the contact point. Since the output of the result takes into account the features of the spatial dimension, the robustness of the detection can be improved.

[0060] The above combination Figure 1 This application provides a detailed description of a contact point detection method for a pin-feed connector based on visual neural networks, as described in embodiments of this application. The following details a contact point detection device for a pin-feed connector based on visual neural networks, used to execute the contact point detection method provided in the embodiments of this application. This device is applied to an electronic device and is configured to perform the above-described method. Figure 1 The method shown.

[0061] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Exemplarily, the electronic device may be a terminal device, or a chip (system) or other component or assembly that can be disposed in the terminal device. Figure 5 As shown, the electronic device 400 may include a processor 401. Optionally, the electronic device 400 may also include a memory 402 and / or a transceiver 403. The processor 401 is coupled to the memory 402 and the transceiver 403, for example, they can be connected via a communication bus. Alternatively, the electronic device 400 may also be a chip, such as including the processor 401; in this case, the transceiver may be the chip's input / output interface.

[0062] The following is combined with Figure 5 The various components of electronic device 400 are described in detail below:

[0063] The processor 401 is the control center of the electronic device 400. It can be a single processor or a collective term for multiple processing elements. For example, the processor 401 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0064] Optionally, the processor 401 can perform various functions of the electronic device 400, such as the aforementioned functions, by running or executing software programs stored in the memory 402 and calling scientific data stored in the memory 402. Figure 1 The method shown is a contact point detection method for pin-feed connectors based on visual nerves.

[0065] In a specific implementation, as one example, processor 401 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 are shown in the diagram.

[0066] In a specific implementation, as one example, the electronic device 400 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process scientific data (such as computer programs or instructions).

[0067] The memory 402 is used to store the software program that executes the solution of this application, and is controlled by the processor 401 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0068] Optionally, the memory 402 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or scientific data structures and accessible by a computer, but not limited thereto. The memory 402 may be integrated with the processor 401 or may exist independently and be accessible through the interface circuit of the electronic device 400. Figure 5 (Not shown in the image) is coupled to processor 401, but this embodiment does not specifically limit this.

[0069] Transceiver 403 is used for communication with other electronic devices. For example, if electronic device 400 is a terminal device, transceiver 403 can be used to communicate with a network device or with another terminal device. As another example, if electronic device 400 is a network device, transceiver 403 can be used to communicate with a terminal device or with another network device.

[0070] Alternatively, transceiver 403 may include a receiver and a transmitter. Figure 5 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0071] Alternatively, the transceiver 403 can be integrated with the processor 401, or it can exist independently and be connected via the interface circuit of the electronic device 400. Figure 5 (Not shown in the image) is coupled to processor 401, but this embodiment does not specifically limit this.

[0072] Understandable Figure 5 The structure of the electronic device 400 shown does not constitute a limitation on the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0073] Furthermore, the technical effects of the electronic device 400 can be referred to the technical effects of the methods described in the above method embodiments, and will not be repeated here.

[0074] It should be understood that the processor in the embodiments of this application can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0075] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0076] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or scientific data center to another website, computer, server, or scientific data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a scientific data storage device such as a server or scientific data center that contains one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0077] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0078] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0079] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0080] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0081] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0084] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0085] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting contact points of a pin-type power connector based on visual nerve perception, characterized in that, Applied to electronic devices, the method includes: The electronic device acquires at least two images, which are images obtained by the imaging device taking pictures of the contact point of the pin-feed connector from different shooting directions, and each of the at least two images contains the contact point; The electronic device uses a visual neural network model to correlate the at least two images in a spatial dimension to obtain the processing result output by the visual neural network model. The processing result indicates whether there is a poor contact at the contact point. The at least two images include a first image and a second image. The processing of the first image in the visual neural network model passes through a first fully connected layer of the visual neural network model, and the processing of the second image in the visual neural network model passes through a second fully connected layer of the visual neural network model. The first and second fully connected layers share some neurons for implementing the association processing in the spatial dimension. Furthermore, the neurons in the first and second fully connected layers form a cellular network structure, and the shared neurons in the cellular network structure are: In the cellular network structure, neurons in the first fully connected layer that are not shared by the second fully connected layer, and neurons in the second fully connected layer that are not shared by the first fully connected layer, are not directly connected to each other. The visual neural network model is configured such that when processing the first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated. And when the visual neural network model processes the second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are activated, and the neurons shared by the first and second fully connected layers are always activated when the visual neural network model processes the first and second images; the first fully connected layer is configured to train to convergence using a training image set obtained by photographing the contact point with the imaging device in a first shooting direction, and the second fully connected layer is configured to train to convergence using a training image set obtained by photographing the contact point with the imaging device in a second shooting direction. The image set is trained to convergence; or, the at least two images include a first image, a second image, and a third image, wherein the processing of the first image in the visual neural network model passes through a first fully connected layer of the visual neural network model, the processing of the second image in the visual neural network model passes through a second fully connected layer of the visual neural network model, and the processing of the third image in the visual neural network model passes through a third fully connected layer of the visual neural network model, wherein the first fully connected layer and the second fully connected layer share some neurons, and the second fully connected layer and the third fully connected layer share some neurons to implement the association processing in the spatial dimension;Furthermore, the neurons included in the first fully connected layer, the neurons included in the second fully connected layer, and the neurons included in the second fully connected layer form a cellular network structure. The neurons shared by the first and second fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the first fully connected layer that are not shared by the second fully connected layer, and also directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer. The neurons shared by the second and third fully connected layers are those neurons in the cellular network structure that are directly connected to neurons in the second fully connected layer that are not shared by the third fully connected layer, and also directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer. In the cellular network structure, neurons in the first fully connected layer that are not shared by the second fully connected layer are not directly connected to neurons in the second fully connected layer that are not shared by the first fully connected layer, and neurons in the second fully connected layer that are not shared by the third fully connected layer are not directly connected to neurons in the third fully connected layer that are not shared by the second fully connected layer. The visual neural network model is configured such that: in the visual neural network model... When processing the first image, neurons in the first fully connected layer that are not shared by the second fully connected layer are activated, and neurons in the second fully connected layer that are not shared by the first fully connected layer are not activated. When the visual neural network model processes the second image, neurons in the first fully connected layer that are not shared by the second fully connected layer are not activated, neurons in the third fully connected layer that are not shared by the second fully connected layer are not activated, and neurons in the second fully connected layer that are not shared by both the first and second fully connected layers are activated. And when the visual neural network model processes the third image... Neurons in the second fully connected layer that are not shared by the third fully connected layer are not activated, while neurons in the third fully connected layer that are not shared by the second fully connected layer are activated. The first fully connected layer is configured to train until convergence using a set of training images obtained by photographing the contact point with the imaging device in the first shooting direction. The second fully connected layer is configured to train until convergence using a set of training images obtained by photographing the contact point with the imaging device in the second shooting direction. The second fully connected layer is configured to train until convergence using a set of training images obtained by photographing the contact point with the imaging device in the third shooting direction.

2. The method according to claim 1, characterized in that, The pin-type power connector includes a base and pins. Each pin includes a first end and a second end. The first end of the pin is disposed on the base. The contact point for power extraction between the first end and the second end includes a first contact point and a second contact point. The first contact point and the second contact point are distributed on both sides of the pin. A first reference line is provided to connect the first contact point and the second contact point, and a second reference line is provided to connect the first end and the second end. The first reference line and the second reference line are located on a reference plane. The at least two images include the first image and the second image. The first image is an image obtained by the shooting device taking pictures of the first contact point and the second contact point in a first shooting direction. The first shooting direction and the reference direction are at a first preset angle, which is not 0. The second image is an image obtained by the shooting device taking pictures of the first contact point and the second contact point in a second shooting direction. The second shooting direction and the reference direction are at a second preset angle, which is not 0. The reference direction is a direction perpendicular to the reference plane. The image of the first contact point in the first image is partially the same as the image of the first contact point in the second image; The image of the second contact point in the first image is partially the same as the image of the second contact point in the second image.

3. The method according to claim 2, characterized in that, The electronic device uses a visual neural network model to correlate the at least two images in a spatial dimension, obtaining the processing result output by the visual neural network model, including: The electronic device convolves the first image through the convolutional layer of the visual neural network model to obtain a first set of convolutional vectors. Then, it performs pooling processing on the first set of convolutional vectors through the pooling layer of the visual neural network model to obtain a first set of pooled vectors. Finally, it inputs the first set of pooled vectors into the first fully connected layer of the visual neural network model through the first channel to obtain the first intermediate analysis result output by the first fully connected layer. The electronic device convolves the second image through the convolutional layer of the visual neural network model to obtain a second set of convolutional vectors. Then, it performs pooling processing on the second set of convolutional vectors through the pooling layer of the visual neural network model to obtain a second set of pooled vectors. Finally, it inputs the second set of pooled vectors into the second fully connected layer of the visual neural network model through the second channel to obtain the second intermediate analysis result output by the second fully connected layer. The first fully connected layer and the second fully connected layer share some neurons to implement the association processing in the spatial dimension; The electronic device processes the first intermediate analysis result and the second intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer.

4. The method according to claim 1, characterized in that, The pin-type power connector includes a base and pins. Each pin includes a first end and a second end. The first end of the pin is disposed on the base. The contact point for power extraction between the first end and the second end includes a first contact point and a second contact point. The first contact point and the second contact point are distributed on both sides of the pin. A first reference line is provided to connect the first contact point and the second contact point, and a second reference line is provided to connect the first end and the second end. The first reference line and the second reference line are located on a reference plane. The at least two images include the first image, the second image, and the third image. The first image is an image obtained by the shooting device taking pictures of the first contact point and the second contact point in a first shooting direction. The first shooting direction and the reference direction are at a first preset angle, and the first preset angle is not 0. The second image is an image obtained by the shooting device taking pictures of the first contact point and the second contact point in a second shooting direction. The second shooting direction and the reference direction are in the same direction. The third image is an image obtained by the shooting device taking pictures of the first contact point and the second contact point in a third shooting direction. The third shooting direction and the reference direction are at a second preset angle, which is not 0. The reference direction is a direction perpendicular to the reference plane. The images of the first contact point in the first image, the first contact point in the second image, and the first contact point in the third image are partially the same; The images of the second contact point in the first image, the second contact point in the second image, and the second contact point in the third image are partially identical.

5. The method according to claim 4, characterized in that, The electronic device uses a visual neural network model to correlate the at least two images in a spatial dimension, obtaining the processing result output by the visual neural network model, including: The electronic device convolves the first image through the convolutional layer of the visual neural network model to obtain a first set of convolutional vectors. Then, it performs pooling processing on the first set of convolutional vectors through the pooling layer of the visual neural network model to obtain a first set of pooled vectors. Finally, it inputs the first set of pooled vectors into the first fully connected layer of the visual neural network model through the first channel to obtain the first intermediate analysis result output by the first fully connected layer. The electronic device convolves the second image through the convolutional layer of the visual neural network model to obtain a second set of convolutional vectors. Then, it performs pooling processing on the second set of convolutional vectors through the pooling layer of the visual neural network model to obtain a second set of pooled vectors. Finally, it inputs the second set of pooled vectors into the second fully connected layer of the visual neural network model through the second channel to obtain the second intermediate analysis result output by the second fully connected layer. The electronic device convolves the third image through the convolutional layer of the visual neural network model to obtain a third convolutional vector set. Then, it performs pooling processing on the third convolutional vector set through the pooling layer of the visual neural network model to obtain a third pooling vector set. Finally, it inputs the third pooling vector set into the third fully connected layer of the visual neural network model through the third channel to obtain the third intermediate analysis result output by the third fully connected layer. The first fully connected layer and the second fully connected layer share some neurons, and the second fully connected layer and the third fully connected layer share some neurons to implement the association processing in the spatial dimension. The electronic device processes the first intermediate analysis result, the second intermediate analysis result, and the third intermediate analysis result through the output layer of the visual neural network model to obtain the processing result output by the output layer.

6. A contact point detection device for a pin-type power connector based on visual nerve perception, characterized in that, Applied to an electronic device, the device is configured to perform the method as described in any one of claims 1-5.

Images