Plug-in state detection method and device of connector, electronic equipment and storage medium

Abstract

The invention discloses a method and device for detecting the plugging state of a connector, electronic equipment and a storage medium. The method for detecting the plugging state of the connector comprises the following steps: carrying out feature identification detection on a connector image to obtain an identification detection result; determining the plugging state of the connector according to the identification detection result; wherein the feature identifier is hidden under the condition that the connector is plugged in place. According to the method for detecting the plugging state of the connector provided by the embodiment of the invention, the accuracy of detecting the plugging state of the connector can be improved, and misjudgment is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of battery detection, and in particular to a method, device, electronic device, and storage medium for detecting the plug-in status of a connector. Background Art

[0002] As the link connecting high and low voltage circuits, wiring harnesses are widely used in various products. Whether the connectors are properly plugged in is one of the main issues affecting the wiring harness assembly process.

[0003] In the related art, the plug-in status detection of the docking plug is carried out by obtaining an image of the area where the connector is located and detecting the pixel distribution of the connector in the image to determine the size and shape of the connector, so as to judge whether the connector is plugged in properly based on the detected size and shape of the connector.

[0004] However, if any of the placement angle, shooting angle, or shooting height of the connector changes, the size and shape of the connector detected through the image will also change, causing the detection result of the plug-in status of the docking connector to change, which can easily lead to misjudgment of the plug-in status of the docking connector. Summary of the Invention

[0005] In view of the above problems, the present application provides a method, device, electronic device and storage medium for detecting the plug-in status of a connector, which can improve the accuracy of detecting the plug-in status of the connector and reduce misjudgment.

[0006] In a first aspect, the present application provides a method for detecting the plug-in status of a connector, the method comprising: performing characteristic identification detection of the connector on a connector image to obtain an identification detection result; determining the plug-in status of the connector based on the identification detection result; wherein the characteristic identification is hidden when the connector is plugged in place.

[0007] In the technical solution of the embodiment of the present application, a connector feature identifier is detected using a docking plug image to obtain an identifier detection result. The connector's plug-in state is then determined based on the identifier detection result. The feature identifier is hidden when the connector is properly plugged in. Since the feature identifier is hidden when the connector is properly plugged in, if the connector is properly plugged in, the feature identifier cannot be detected from the connector image, and the connector's plug-in state can be determined to be fully plugged in. Conversely, if at least a portion of the feature identifier is detected from the connector image, the connector's plug-in state can be determined to be partially plugged in. Therefore, plug-in state detection does not require consideration of the connector's size and shape, and the spatial position and angle of the docking plug are not critical. Thus, even if the placement or shooting angle of the connector changes, whether the connector is properly plugged in can be effectively determined based on whether the feature identifier is recognized from the connector image. This allows for compatible plug-in state detection of connectors from multiple angles and in multiple states, thereby improving the accuracy of plug-in state detection and reducing misjudgments.

[0008] In some embodiments, performing connector feature identification detection on a connector image to obtain an identification detection result includes: performing image extraction on the connector image to obtain a connector region within the connector image; and performing connector feature identification detection on the connector region to obtain an identification detection result. This eliminates interference from the connector image's background on the detection result, thereby improving the accuracy and robustness of the connector's plugged-in status detection result and reducing false positives.

[0009] In some embodiments, performing image extraction on the connector image to obtain the connector region in the connector image includes: performing image extraction on the connector image using an image detection model to obtain the connector region in the connector image; wherein the image detection model is trained using sample connector images. In this manner, the image detection model can be used to accurately extract the connector region from the connector image, thereby improving the accuracy of subsequent detection.

[0010] In some embodiments, performing image extraction on the connector image to obtain a connector region within the connector image includes: performing edge extraction on the connector image to obtain an edge image of the connector image; and obtaining the connector region within the connector image based on the edge image of the connector image. Thus, by extracting the connector region from the connector image through edge extraction, interference from the background of the connector image and the docking connector image is effectively removed, thereby improving the accuracy and robustness of subsequent docking status detection and reducing false positives.

[0011] In some embodiments, performing connector feature identification detection on a connector image to obtain an identification detection result includes: performing connector feature identification detection on a connector region within the connector image based on the color value of the feature identification to obtain the identification detection result. This allows obtaining an identification detection result by simply identifying the color values ​​of pixels within the connector region, without having to consider the shape of the feature identification or the location of the pixels within the feature identification, thereby improving feature identification detection efficiency.

[0012] In some embodiments, determining the plug-in state of the connector based on the identification detection result includes: determining that the identification detection result is that the image area of ​​the characteristic identification is less than or equal to a preset area, and detecting the connector area in the connector image; determining the pixel points representing the connector connection point identified from the connector area, and determining the plug-in state of the connector based on the identification detection result, so as to avoid the situation where the characteristic identification is blocked by foreign matter, resulting in the connector plug-in state being misjudged as being plugged in due to the failure to identify the characteristic identification in the connector area, thereby improving the accuracy of the plug-in state detection result.

[0013] In some embodiments, the method further includes: determining that no pixel representing a connector connection point is identified in the connector area, and generating prompt information indicating that there is a foreign object blocking the connector image.

[0014] In some embodiments, the color value of the feature identifier is different from the color value of the body of the connector, so as to reduce the interference caused by the color of the connector on the feature identifier detection, thereby greatly improving the detection success rate and the accuracy of the obtained identifier detection results through the color difference after the connector is in place.

[0015] In a second aspect, the present application provides a device for detecting the plug-in status of a connector, comprising: a feature identification detection module for performing feature identification detection of the connector on a connector image to obtain an identification detection result; a plug-in status detection module for determining the plug-in status of the connector based on the identification detection result; wherein the feature identification is hidden when the connector is plugged in place.

[0016] In the technical solution of the embodiment of the present application, the present solution detects the characteristic identifier of the connector using the docking plug image to obtain an identifier detection result, and then determines the plug-in state of the connector based on the identifier detection result. The characteristic identifier is hidden when the connector is plugged in. Since the characteristic identifier is hidden when the connector is plugged in, if the connector is plugged in, the characteristic identifier cannot be detected from the connector image, and the plug-in state of the connector can be determined to be fully plugged in. Conversely, if at least part of the characteristic identifier is detected from the connector image, the plug-in state of the connector can be determined to be partially plugged in. Therefore, the plug-in state detection does not need to consider the size and shape of the connector, and the spatial position and angle of the docking plug are not high. In this way, even if the placement position or shooting angle of the connector changes, whether the connector is plugged in can be effectively determined by whether the characteristic identifier is recognized from the connector image. This allows for compatibility with multi-angle and multi-state plug-in state detection of the connector, thereby improving the accuracy of plug-in state detection and reducing misjudgments.

[0017] In some embodiments, the feature identification detection module is specifically used to: perform image extraction on the connector image to obtain a connector area in the connector image; perform connector feature identification detection on the connector area to obtain an identification detection result.

[0018] In some embodiments, the feature identification detection module is specifically used to: use an image detection model to perform image extraction on the connector image to obtain a connector area in the connector image; wherein the image detection model is trained by various connector sample images.

[0019] In some embodiments, the feature identification detection module is specifically configured to: perform edge extraction on the connector image to obtain an edge image of the connector image; and obtain a connector region in the connector image based on the edge image of the connector image.

[0020] In some embodiments, the feature identification detection module is specifically configured to perform feature identification detection of the connector on the connector region in the connector image according to the color value of the feature identification to obtain an identification detection result.

[0021] In some embodiments, the plug-in status detection module is specifically used to: determine that the image area of ​​the feature identification result is less than or equal to a preset area, and detect the connector area in the connector image; determine the pixel points representing the connector connection point identified from the connector area, and determine the plug-in status of the connector based on the identification detection result.

[0022] In some embodiments, the plug-in status detection module is further configured to: determine that no pixel representing a connector connection is identified in the connector area, and generate prompt information indicating that there is a foreign object blocking the connector image.

[0023] In some embodiments, the color value of the feature identifier is different from the color value of the body of the connector.

[0024] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the method in the implementation of the first aspect when executing the computer program.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the method in the embodiment of the first aspect.

[0026] In a fifth aspect, the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method in the first aspect, any optional implementation method in the first aspect, or the third aspect, any optional implementation method in the third aspect.

[0027] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference numerals are used throughout the drawings to represent the same components. In the drawings:

[0029] Figure 1 A flowchart of a method for detecting the plug-in status of a connector according to some embodiments of the present application;

[0030] Figure 2a A planar structural diagram of a connector according to some embodiments of the present application;

[0031] Figure 2b A three-dimensional structural diagram of a connector according to some embodiments of the present application;

[0032] Figure 3a This is a planar structural diagram of the connectors of some embodiments of the present application when they are fully plugged in;

[0033] Figure 3b This is a three-dimensional structural diagram of the connectors of some embodiments of the present application when they are fully plugged in;

[0034] Figure 4a This is a planar structural diagram of a connector in some embodiments of the present application when it is not fully plugged in;

[0035] Figure 4b This is a three-dimensional structural diagram of a connector in some embodiments of the present application when it is not fully connected;

[0036] Figure 5 This is a schematic diagram of foreign body obstruction in some embodiments of the present application;

[0037] Figure 6 This is a schematic structural diagram of a device for detecting the plug-in status of a connector in some embodiments of the present application;

[0038] Figure 7 This is a schematic structural diagram of an electronic device according to some embodiments of the present application.

[0039] The accompanying drawings in the specific implementation manner are as follows:

[0040] 10 - connector; 20 - pair of connectors; 100 - feature identification; 200 - foreign matter; 301 - feature identification detection module; 302 - plug-in status detection module; 40 - electronic device; 401 - processor; 402 - memory; 403 - communication bus. DETAILED DESCRIPTION

[0041] The following embodiments of the technical solution of the present application will be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application and are therefore only examples and are not intended to limit the scope of protection of the present application.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0043] In the description of the embodiments of this application, the technical terms "first" and "second" are used only to distinguish different objects and should not be understood to indicate or imply relative importance or implicitly specify the quantity, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, the meaning of "plurality" is more than two, unless otherwise clearly and specifically defined.

[0044] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0045] In the description of the embodiments of this application, the term "and / or" is simply a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0046] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0047] Wiring harnesses, the link between high and low voltage circuits, are widely used in various products. Currently, wiring harnesses are typically connected through plug-in connectors. Improper insertion of connectors can affect the wiring harness assembly process and, consequently, the normal use of the product. Therefore, it is necessary to inspect the plug-in status of the connectors.

[0048] Currently, the plug-in status of a docking connector can be detected by capturing an image of the area where the connector is located and detecting the pixel distribution of the connector in the image to determine the size and shape of the connector. Based on the size and shape of the connector, the connector can be judged as properly plugged in. For example, the size and shape of the connector are matched with a standard assembly template corresponding to the connector. If the identified size and shape of the connector match the standard assembly template, the connector can be determined to be properly plugged in; otherwise, the connector can be determined to be improperly plugged in. However, if any of the placement angle, camera angle, or camera height of the connector changes, the size and shape of the connector detected through the image will also change. For example, if the placement angle of the connector is tilted, the size and shape of the connector detected through the image will change, and some structures may even be incomplete, thus changing the plug-in status detection result of the docking connector. Therefore, this detection method can easily lead to misjudgment of the docking connector's plug-in status.

[0049] To address the above technical issues, embodiments of the present application provide a method for detecting the plug-in state of a connector. This method detects a characteristic identifier of the connector from a connector image to obtain an identifier detection result, and then determines the plug-in state of the connector based on the identifier detection result. The characteristic identifier is hidden when the connector is fully plugged in. Since the characteristic identifier is hidden when the connector is fully plugged in, if the connector is fully plugged in, the characteristic identifier cannot be detected from the connector image, and the plug-in state of the connector can be determined to be fully plugged in. Conversely, if at least a portion of the characteristic identifier is detected from the connector image, the plug-in state of the connector can be determined to be partially plugged in. Therefore, plug-in state detection does not require consideration of the size and shape of the connector, and the spatial position and angle of the connector are not critical. In this way, even if the placement position or shooting angle of the connector changes, whether the connector is fully plugged in can be effectively determined by whether the characteristic identifier is recognized from the connector image. This allows for multi-angle and multi-state connector plug-in state detection, thereby improving the accuracy of connector plug-in state detection and reducing false positives.

[0050] The connector plug-in status detection method, device, electronic device, and storage medium disclosed in the embodiments of the present application can be applied to a terminal device to detect the plug-in status of the connector. The terminal device can be a desktop terminal, a mobile terminal, or a server. The server can be an independent server or a server cluster composed of multiple servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence sampling point devices.

[0051] According to some embodiments of the present application, taking a method for detecting the plugging status of a connector provided in an embodiment of the present application as an example, the plugging status detection method can be applied to the aforementioned terminal device.

[0052] like Figure 1 As shown, the method for detecting the plug-in status of the connector includes:

[0053] S101, performing feature identification detection on the plug-in image to obtain an identification detection result;

[0054] S102, determining the plug-in status of the connector according to the identification detection result;

[0055] Wherein, the characteristic mark is hidden when the connector is plugged into place.

[0056] In some embodiments, the plug portion of the connector may be pre-set with a characteristic mark, which may be a pattern or a color mark, such as a dot, line, surface or other pattern with at least one specific color. Figure 2a As shown, the connector 10 may be sprayed with a characteristic mark 100, which may be any color or a color block that is opposite to the color of the connector. For ease of detection, the characteristic mark 100 may be sprayed on all sides of the connector 10, such as a color block that surrounds the connector 10. Figure 2b As shown, this is to avoid the situation where a certain face is blocked and the unhidden feature identifier cannot be identified.

[0057] The connector 10 is used to connect to the plug-in 20. If the connector 10 is plugged into place, the characteristic mark of the connector 10 will be completely covered by the plug-in 20, thereby hiding the characteristic mark. Figure 3a or Figure 3b If the connector 10 is not connected to the plug-in 20 or is not properly plugged in, the characteristic mark of the connector 10 will not be fully covered by the plug-in 20, and there may be a situation where at least part of the mark is exposed, such as Figure 4a or Figure 4b shown.

[0058] When inspecting the plugged-in status of a connector, a visual inspection device, such as a video camera or a color CCD camera, can be used to capture an image of the connector in the area where the connector is placed. The visual inspection device can have a wide field of view, meaning a wide imaging range, so that the captured image can include multiple connectors. This allows for simultaneous inspection of multiple connectors, improving inspection efficiency.

[0059] After acquiring the connector image, a feature identifier detection can be performed on the connector image to obtain a detection result for the feature identifier. For example, based on the pixels of the feature identifier, a detection can be performed to determine whether there are any matching pixels in the acquired connector image. If there are matching pixels in the connector image, it indicates that the feature identifier has been identified from the connector image, and the detection result can be determined to be that the feature identifier is not hidden. If there are no matching pixels in the connector image, it indicates that the feature identifier is not present in the connector image, and the detection result can be determined to be that the feature identifier is hidden.

[0060] Alternatively, the image area of ​​the characteristic identifier in the connector image can be directly obtained based on the pixel matching results. This image area is the identifier detection result. For example, all pixels in the connector image that match the pixels of the characteristic identifier are obtained to form the image area of ​​the characteristic identifier in the connector image. It will be understood that if there are no pixels in the connector image that match the pixels of the characteristic identifier, the image area of ​​the characteristic identifier is zero.

[0061] As a possible implementation method, after obtaining the identification detection result, if the identification detection result is that the characteristic identification exists in the connector image, it can be determined that the characteristic identification is not completely covered by the connector's counterpart, and at this time, the plug-in state of the connector can be determined as not plugged in; if the identification detection result is that the characteristic identification does not exist in the connector image, it can be determined that the characteristic identification is completely covered by the connector's counterpart, and at this time, the plug-in state of the connector can be determined as plugged in.

[0062] As a possible implementation, if the identification detection result is the image area of ​​the characteristic identification, this image area can be compared with a preset area. If the image area of ​​the characteristic identification is larger than the preset area, the connector can be determined to be not properly connected; otherwise, the connector can be determined to be properly connected. For example, the preset area can be set to zero. If the image area of ​​the characteristic identification is larger than zero, the connector can be determined to be not properly connected; otherwise, the connector can be determined to be properly connected. Taking into account the possible offset of the connector during insertion, the preset area can also be set to the minimum area of ​​the characteristic identification identified from multiple shooting angles when the connector is properly connected.

[0063] The above-designed method for detecting the plug-in state of a connector detects a characteristic identifier of the connector using a docking connector image to obtain an identifier detection result. The connector's plug-in state is then determined based on the identifier detection result. The characteristic identifier is hidden when the connector is properly plugged in. Since the characteristic identifier is hidden when the connector is properly plugged in, if the connector is properly plugged in, the characteristic identifier cannot be detected from the connector image, and the connector's plug-in state can be determined to be fully plugged in. Conversely, if at least a portion of the characteristic identifier is detected from the connector image, the connector's plug-in state can be determined to be partially plugged in. Therefore, plug-in state detection does not require consideration of the connector's size and shape, and the spatial position and angle of the docking connector are not critical. Thus, even if the placement or camera angle of the connector changes, whether the connector is properly plugged in can be effectively determined based on whether the characteristic identifier is recognized from the connector image. This allows for multi-angle and multi-state connector plug-in state detection, thereby improving the accuracy of plug-in state detection and reducing misjudgments.

[0064] Considering that the background in the connector image may interfere with detection, for example, if there is a color block in the background of the connector image that is the same color as the characteristic identification, it may lead to misjudgment. Therefore, in order to reduce the impact of the background of the connector image on the detection results and further improve the accuracy of the plug-in state detection, in some embodiments, the connector characteristic identification detection is performed on the connector image to obtain the identification detection result, including:

[0065] Perform image extraction on the connector image to obtain a connector area in the connector image;

[0066] Perform feature identification detection of the connector on the connector area to obtain an identification detection result.

[0067] In some embodiments, after acquiring a connector image, image extraction can be performed on the connector image through image recognition to extract the connector region from the connector image. For example, a sample image containing only an unplugged connector and a sample image of a plugged connector can be matched with the connector image to extract the connector region from the connector image. Alternatively, the connector position in the connector image can be located based on pre-set feature points of the unplugged connector and pre-plugged connector to extract the connector region from the connector image.

[0068] After extracting the connector area from the connector image, the features of the connector area can be identified to obtain the identification detection result of the connector area. This eliminates the interference of the connector image background on the detection result, thereby improving the accuracy and robustness of the connector plug-in status detection result and reducing false positives.

[0069] To more accurately extract the connector area and further improve the accuracy of the detection result, in some embodiments, performing image extraction on the connector image to obtain the connector area in the connector image includes:

[0070] Perform image extraction on the connector image using the image detection model to obtain the connector area in the connector image;

[0071] The image detection model is trained by using sample images of various connectors.

[0072] In some embodiments, the image detection model can be a trained neural network, such as any one of a convolutional neural network, U-Net, Mask RCNN, Deep Lab v3+, or FCIS (Fully Convolutional Instance-aware Semantic Segmentation) neural network.

[0073] The image detection model can be trained using sample connector images. These sample connector images include sample images of an unplugged connector taken at various angles, and sample images of a plugged connector taken at various angles. Each sample connector image can include both the connector area and a background image.

[0074] As a possible implementation, each sample connector image can be sequentially input into the image detection model. After obtaining the connector region identification output by the image detection model for each input, the identified connector region is then matched with the preset connector region corresponding to the currently input connector sample image. If the two regions do not match, a gradient descent method is used to adjust the network parameters of the image detection model through error backpropagation. Training is then repeated until the connector region identified from each newly input connector sample image matches the preset connector region corresponding to the currently input connector sample image. This completes the image detection model training, resulting in a pre-trained image detection model.

[0075] After obtaining a pre-trained image detection model, the acquired connector image can be input into the image detection model to identify the connector area from the connector image. This allows the image detection model to accurately extract the connector area from the connector image, thereby improving the accuracy of subsequent detection.

[0076] In addition to extracting the connector region from the connector image using an image detection model, in some embodiments, performing image extraction on the connector image to obtain the connector region in the connector image includes:

[0077] Perform edge extraction on the connector image to obtain an edge image of the connector image;

[0078] A connector region in the connector image is obtained according to the edge image of the connector image.

[0079] In some embodiments, edge extraction refers to the processing of image contours in digital image processing. The point where the grayscale change rate of an image is the largest is defined as an edge, also known as an inflection point. An inflection point is a point where the function undergoes a change in concavity and convexity, and where the second-order derivative is zero. Edge extraction of the plug-in image can be performed using an edge detection operator to extract the contours of all objects from the plug-in image to form an edge image.

[0080] The edge detection operator can be any one of a gradient edge operator, a Roberts edge operator, a Laplace edge operator, and a Sobel edge operator. As a possible implementation, considering that the connector image obtained by photographing the connector is typically a step-edge image, and the Sobel edge operator is good at edge extraction in step-edge images, has a certain noise suppression capability, and relatively accurate and complete edge positioning, the image edge of the connector image can be extracted by approximating the first-order differential Sobel edge operator in the x-direction and / or y-direction to obtain an edge image of the connector image.

[0081] After acquiring the edge image, connector boundary detection can be performed on the edge image to extract the connector region from the connector image based on the detected connector boundary. For example, an edge line corresponding to the outline of an unplugged connector, or the outline of the connector plugged into its mating partner, can be identified from the edge image as the connector boundary. The identified connector boundary can then be used to extract the connector region from the connector image. This edge extraction effectively removes interference from the background image and the mating partner image, thereby improving the accuracy and robustness of subsequent detection of the mating partner's plugged-in state and reducing false positives.

[0082] After extracting the connector region from the connector image, connector feature identification detection can be performed on the connector region. To improve feature identification detection efficiency, in some embodiments, connector feature identification detection can be performed on the connector region in the connector image based on the color value of the feature identification to obtain an identification detection result.

[0083] The color value refers to the intensity value in the range of 0 to 255 assigned to the RGB components of a pixel in the RGB color model. Since the color value of the feature identifier is known, for example, if the feature identifier is a red block, its color value can be determined to be (255, 0, 0). Therefore, the color value of the feature identifier can be used to traverse the connector area and all pixels with a color value of (255, 0, 0) identified in the connector area are used as the identification detection result for the connector area.

[0084] The color value of the feature identifier is used to detect the feature identifier of the connector in the connector area in the connector image. Therefore, the identifier detection result can be obtained by only identifying the color value of the pixels in the connector area, without considering the shape of the feature identifier and the position of the pixel points of the feature identifier, thereby improving the detection efficiency of the feature identifier.

[0085] In order to reduce the interference of the color of the connector on the detection of the characteristic identifier, in some embodiments, the color value of the characteristic identifier may be different from the color value of the body of the connector, that is, the color of the characteristic identifier is different from that of the body of the connector. For example, the characteristic identifier can be a color block that is opposite to the color of the connector, so that the characteristic identifier can be identified more efficiently. Exemplarily, the color value of the characteristic identifier and the color value of the connector body belong to different sets of color values. For example, assuming that the body of the connector is white, the color identifier can be red, that is, the color value of the body of the connector is one of the color value sets that belong to the color category of white, such as the color value corresponding to one of the white colors such as milky white, ivory white, and beige, and the color value of the color identifier is one of the color value sets that belong to the color category of red, such as the color value corresponding to one of the red colors such as fiery red, pink, and magenta. In this way, since there is a color difference between the characteristic identification of the connector and the main body of the connector, the main body of the connector will not be mistakenly identified as the characteristic identification during the detection of the characteristic identification in the docking connector area, thereby reducing the interference of the color of the connector on the characteristic identification detection, and through the color difference after the connector is in place, the detection success rate is greatly improved, and the accuracy of the obtained identification detection results is improved.

[0086] After obtaining the identification detection result, the plug-in state of the connector can be determined based on the identification detection result. For example, assuming that the identification detection result shows that the image area of ​​the characteristic identification is less than or equal to the preset area, such as the image area of ​​the characteristic identification is zero, the plug-in state of the connector can be determined to be fully plugged in. If the identification detection result shows that the image area of ​​the characteristic identification is greater than the preset area, such as the image area of ​​the characteristic identification is not zero, the plug-in state of the connector can be determined to be not fully plugged in.

[0087] However, in actual production, due to environmental influences, the connector area may be partially blocked by the wiring harness or other foreign objects. This may result in the connector not being properly plugged in, but the characteristic mark being blocked by the foreign object. This may lead to the connector's plug-in status being misjudged as being properly plugged in because the characteristic mark is not recognized in the connector area. Therefore, to further reduce misjudgments, in some embodiments, determining the connector's plug-in status based on the mark detection results includes:

[0088] Determining that the image area of ​​the characteristic mark as a result of the mark detection is less than or equal to a preset area, and detecting a connector area in the connector image;

[0089] Determine the pixel points representing the connector connection points identified in the connector area, and determine the plug-in state of the connector based on the identification detection result.

[0090] In some embodiments, the characteristic mark can be a color block surrounding the connector, such as Figure 2bAs shown. The connector connection point can be the location where the connector connects to the plug-in, such as the plug of the connector, or the intersection of the connector and the plug-in. For example, if the connector is connected to the plug-in, there will be a boundary line between the connector and the plug-in in the connector area. Figure 5 As shown, if the boundary line is not completely obscured by foreign object 200, the boundary line can still be identified from the connector area. Furthermore, since the characteristic marker can be a color block surrounding the connector, if the boundary line is not completely obscured by the foreign object, a portion of the characteristic marker can still be identified from the connector area. Therefore, if the marker detection result indicates that the image area of ​​the characteristic marker is less than or equal to the preset area, it is possible to detect whether there are pixels belonging to the boundary line in the connector area.

[0091] If a pixel containing this boundary line is detected in the connector area, it indicates that the characteristic marker is not completely obscured by the foreign object. In this case, the connector can be determined to be properly plugged in based on the marker detection result. This avoids the situation where the characteristic marker is obscured by the foreign object, resulting in the connector being misjudged as properly plugged in due to the failure to recognize the characteristic marker in the connector area, thereby improving the accuracy of the plug-in state detection result.

[0092] Even if there is obstruction by foreign matter, as long as the foreign matter does not completely block the connection of the connector, the plug-in status of the connector can still be determined by the identification detection result, so it can be applied to plug-in status detection in more environments.

[0093] In some embodiments, if it is determined that no pixels representing the connector connection are identified in the connector area, a prompt message indicating that a foreign object is obstructing the connector image is generated. For example, if the identification detection result indicates that the image area of ​​the feature identifier is less than or equal to a predetermined area, and if no pixels representing the boundary between the connector and the mating connector are identified in the connector area, it can be determined that the feature identifier is likely completely obstructed by a foreign object. In this case, a prompt message indicating that a foreign object is obstructing the connector image can be generated and sent to the user terminal, prompting the user to remove the corresponding foreign object.

[0094] In some embodiments, if the connector is determined to be properly plugged in, a prompt message indicating that the connector is properly plugged in may be generated. If the connector is determined to be improperly plugged in, a prompt message indicating that the connector is abnormally plugged in may be generated and sent to the user terminal, prompting the user to re-inspect or repair the connector.

[0095] Figure 6 The present application provides a schematic diagram of the structure of a connector plug-in status detection device. It should be understood that the device is Figure 1The method embodiment executed in the embodiment corresponds to the method, and the steps involved in the aforementioned method can be executed. The specific functions of the device can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in the memory in the form of software or firmware or solidified in the operating system (OS) of the device. Specifically, the device includes: a feature identification detection module 301, which is used to perform feature identification detection of the connector on the plug-in image to obtain an identification detection result; a plug-in status detection module 302, which is used to determine the plug-in status of the connector based on the identification detection result; wherein the feature identification is hidden when the connector is plugged in.

[0096] In the technical solution of the embodiment of the present application, a connector feature identifier is detected using a docking plug image to obtain an identifier detection result. The connector's plug-in state is then determined based on the identifier detection result. The feature identifier is hidden when the connector is properly plugged in. Since the feature identifier is hidden when the connector is properly plugged in, if the connector is properly plugged in, the feature identifier cannot be detected from the connector image, and the connector's plug-in state can be determined to be fully plugged in. Conversely, if at least a portion of the feature identifier is detected from the connector image, the connector's plug-in state can be determined to be partially plugged in. Therefore, plug-in state detection does not require consideration of the connector's size and shape, and the spatial position and angle of the docking plug are not critical. Thus, even if the placement or shooting angle of the connector changes, whether the connector is properly plugged in can be effectively determined based on whether the feature identifier is recognized from the connector image. This allows for compatible plug-in state detection of connectors from multiple angles and in multiple states, thereby improving the accuracy of plug-in state detection and reducing misjudgments.

[0097] According to some embodiments of the present application, the feature identification detection module 301 is specifically used to: perform image extraction on the connector image to obtain a connector area in the connector image; perform connector feature identification detection on the connector area to obtain an identification detection result.

[0098] According to some embodiments of the present application, the feature identification detection module 301 is specifically used to: use an image detection model to extract the image of the connector to obtain the connector area in the connector image; wherein the image detection model is obtained by training each connector sample image.

[0099] According to some embodiments of the present application, the feature identification detection module 301 is specifically used to: perform edge extraction on the connector image to obtain an edge image of the connector image; and obtain a connector area in the connector image based on the edge image of the connector image.

[0100] According to some embodiments of the present application, the feature identification detection module 301 is specifically configured to perform feature identification detection of the connector on the connector region in the connector image according to the color value of the feature identification to obtain an identification detection result.

[0101] According to some embodiments of the present application, the plug-in status detection module 302 is specifically used to: determine that the image area of ​​the feature identification result is less than or equal to a preset area, and detect the connector area in the connector image; determine the pixel points representing the connector connection point identified from the connector area, and determine the plug-in status of the connector based on the identification detection result.

[0102] According to some embodiments of the present application, the plug-in status detection module 302 is further configured to: determine that no pixel representing a connector connection is identified in the connector area, and generate prompt information indicating that there is foreign matter blocking the connector image.

[0103] According to some embodiments of the present application, the color value of the characteristic identifier is different from the color value of the body of the connector.

[0104] According to some embodiments of the present application, Figure 7 As shown, the present application provides an electronic device 40, including: a processor 401 and a memory 402, the processor 401 and the memory 402 are interconnected and communicate with each other through a communication bus 403 and / or other forms of connection mechanisms (not shown), and the memory 402 stores a computer program executable by the processor 401. When the computing device is running, the processor 401 executes the computer program to execute the method executed by the external terminal in any optional implementation method, for example: performing feature identification detection of the connector on the docking plug image to obtain an identification detection result; determining the plug-in status of the connector based on the identification detection result; wherein the feature identification is hidden when the connector is plugged in.

[0105] The present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method in any of the aforementioned optional implementations is executed.

[0106] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination of them, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0107] The present application provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the method in any optional implementation manner.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application. In particular, as long as there is no structural conflict, the various technical features mentioned in the various embodiments can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A method for detecting the plug-in status of a connector, characterized in that: include: Performing feature identification detection on the connector image to obtain an identification detection result; determining the plug-in status of the connector according to the identification detection result; Wherein, the characteristic mark is hidden when the connector is plugged into place.

2. The method according to claim 1, characterized in that Perform feature identification detection on the connector image to obtain identification detection results, including: performing image extraction on the connector image to obtain a connector region in the connector image; A feature identification detection of the connector is performed on the connector area to obtain an identification detection result.

3. The method according to claim 2, characterized in that Performing image extraction on the connector image to obtain a connector region in the connector image includes: Performing image extraction on the connector image using an image detection model to obtain a connector region in the connector image; The image detection model is obtained by training sample images of various connectors.

4. The method according to claim 2, characterized in that Performing image extraction on the connector image to obtain a connector region in the connector image includes: performing edge extraction on the connector image to obtain an edge image of the connector image; A connector region in the connector image is obtained according to the edge image of the connector image.

5. The method according to any one of claims 1 to 4, characterized in that: Perform feature identification detection on the connector image to obtain identification detection results, including: According to the color value of the characteristic identification, the characteristic identification detection of the connector is performed on the connector area in the connector image to obtain an identification detection result.

6. The method according to any one of claims 1 to 5, characterized in that: Determining the plug-in state of the connector according to the identification detection result includes: Determining that the identification detection result is that the image area of ​​the characteristic identification is less than or equal to a preset area, and detecting a connector area in the connector image; A pixel point representing a connector connection position is identified from the connector area, and a plug-in state of the connector is determined based on the identification detection result.

7. The method according to claim 6, characterized in that Also includes: It is determined that no pixel representing a connector connection point is identified in the connector area, and prompt information is generated to indicate that there is a foreign object blocking the connector image.

8. The method according to any one of claims 1 to 6, characterized in that: The color value of the characteristic mark is different from the color value of the body of the connector.

9. A device for detecting the plug-in status of a connector, characterized in that: The device comprises: A feature identification detection module is used to detect the feature identification of the connector based on the connector image and obtain an identification detection result; A plug-in status detection module, configured to determine the plug-in status of the connector according to the identification detection result; Wherein, the characteristic mark is hidden when the connector is plugged into place.

10. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images