Device accessory detection method, apparatus, system, device, and storage medium

Abstract

The application provides a device accessory detection method, device, system, equipment and storage medium. The accessory detection method is applied to an edge computing node, and the edge computing node is preloaded with an accessory identification network; the method comprises the following steps: obtaining a target accessory image of a device under test; identifying the target accessory image by using the accessory identification network to obtain a predicted classification result of the target accessory; comparing the predicted classification result with reference type information of a target accessory corresponding to the device under test to obtain a detection result of the target accessory. The technical solution provided by the application improves the accessory detection efficiency and reduces the false detection rate and the missed detection rate.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, system, device, and storage medium for detecting equipment accessories. Background Technology

[0002] For products or equipment that assemble multiple parts, the parts need to be checked after the assembly process is completed to reduce quality defects such as incorrect assembly or omissions, thereby improving product quality.

[0003] Traditional equipment parts inspection processes are generally carried out manually, which is inefficient and prone to omissions and errors. Summary of the Invention

[0004] In view of this, in order to solve the above-mentioned technical problems, this application provides a method, apparatus, system, equipment and storage medium for testing equipment accessories.

[0005] Specifically, this application is implemented through the following technical solution:

[0006] According to a first aspect of the embodiments of this application, a method for detecting equipment accessories is provided, the method being applied to an edge computing node, the edge computing node being pre-loaded with an accessory identification network; the method includes:

[0007] Acquire images of the target components of the device under test;

[0008] The target accessory image is identified using the accessory recognition network to obtain the predicted classification result of the target accessory;

[0009] The predicted classification result is compared with the reference type information of the target accessory corresponding to the tested device to obtain the detection result of the target accessory.

[0010] Optionally, acquiring the target accessory image of the device under test includes:

[0011] Acquire an image of the device under test;

[0012] The image is detected using a target detection network to obtain an image region containing the target accessory;

[0013] The target accessory image is obtained based on the image region.

[0014] Optionally, the device under test includes a vehicle; acquiring an image of the device under test includes:

[0015] Images of multiple parts of the vehicle are acquired using multiple cameras positioned around the vehicle.

[0016] Optionally, the step of using the accessory recognition network to recognize the target accessory image and obtain the predicted classification result of the target accessory includes:

[0017] Obtain the probability values ​​of the target accessory belonging to multiple preset categories;

[0018] The maximum value among the multiple preset classification probability values ​​is determined, and the maximum value is compared with the first classification threshold corresponding to the target accessory. If the maximum value is greater than or equal to the first classification threshold corresponding to the target accessory, the type corresponding to the maximum value is determined as the predicted classification result of the target accessory.

[0019] Optionally, the method further includes:

[0020] For the accessory recognition network, the recall and precision corresponding to the target accessory are obtained by setting different classification thresholds;

[0021] The first classification threshold is determined based on the precision-recall PR curves corresponding to the recall and precision.

[0022] Optionally, the method further includes:

[0023] Upon obtaining the detection result of the target accessory of the device under test, the image of the target accessory and the corresponding detection result are uploaded to the cloud so that the cloud can optimize the accessory recognition network.

[0024] The optimized accessory identification network is used to update the accessory identification network.

[0025] Optionally, the method further includes:

[0026] For the optimized accessory recognition network, the recall and precision of the target accessory are obtained by setting different classification thresholds in the cloud.

[0027] The second classification threshold is determined based on the PR curves corresponding to recall and precision, and the first classification threshold is updated to the second classification threshold.

[0028] Optionally, the method further includes:

[0029] In response to a discrepancy between the predicted classification result and the reference type information of the target accessory corresponding to the tested device, a prompt message is sent.

[0030] According to a second aspect of the present application, a device for detecting equipment accessories is provided. The device is applied to an edge computing node, the edge computing node being pre-loaded with an accessory identification network; the device includes:

[0031] The accessory image acquisition module acquires images of the target accessories of the device under test.

[0032] The classification prediction module uses the accessory recognition network to identify the target accessory image and obtain the predicted classification result of the target accessory.

[0033] The information comparison module compares the predicted classification result with the reference type information of the target accessory corresponding to the tested device to obtain the detection result of the target accessory.

[0034] Optionally, the accessory image acquisition module is specifically used for:

[0035] Acquire an image of the device under test;

[0036] The image is detected using a target detection network to obtain an image region containing the target accessory;

[0037] The target accessory image is obtained based on the image region.

[0038] Optionally, the device under test includes a vehicle; acquiring an image of the device under test includes:

[0039] Images of multiple parts of the vehicle are acquired using multiple cameras positioned around the vehicle.

[0040] Optionally, the classification prediction module is specifically used for:

[0041] Obtain the probability values ​​of the target accessory belonging to multiple preset categories;

[0042] The maximum value among the multiple preset classification probability values ​​is determined, and the maximum value is compared with the first classification threshold corresponding to the target accessory. If the maximum value is greater than or equal to the first classification threshold corresponding to the target accessory, the type corresponding to the maximum value is determined as the predicted classification result of the target accessory.

[0043] Optionally, the classification prediction module further includes:

[0044] The first rate value acquisition module is used to acquire the recall and precision corresponding to the target accessory by setting different classification thresholds for the accessory recognition network.

[0045] The threshold determination module is used to determine the first classification threshold based on the precision-recall PR curve corresponding to the recall and precision.

[0046] Optionally, the device further includes:

[0047] The information transmission module is used to upload the image of the target accessory and the corresponding detection result to the cloud after obtaining the detection result of the target accessory of the device under test, so that the cloud can optimize the accessory recognition network;

[0048] The identification network update module is used to update the part identification network using the optimized part identification network.

[0049] Optionally, the identification network update module further includes:

[0050] The second rate value acquisition module is used to obtain the recall and precision of the target accessory by setting different classification thresholds in the cloud for the optimized accessory recognition network.

[0051] The threshold update module is used to determine the second classification threshold based on the PR curves corresponding to the recall and precision, and update the first classification threshold to the second classification threshold.

[0052] Optionally, the device further includes:

[0053] The result response module is used to send a prompt message when the predicted classification result is inconsistent with the reference type information of the target accessory corresponding to the tested device.

[0054] According to a third aspect of the embodiments of this application, a device accessory detection system is provided, the system including an image acquisition device and an edge computing node; the edge computing node is preloaded with an accessory recognition network.

[0055] The image acquisition device is used to acquire images of multiple components of the device under test;

[0056] The edge computing node is used to acquire the target accessory image of the device under test, and to identify the target accessory image using the accessory recognition network to obtain the detection result of the target accessory.

[0057] According to a fourth aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a memory and a processor; the memory being used to store a computer program; the processor being used to execute the above-described device accessory detection method by invoking the computer program.

[0058] According to a fifth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the program, when executed by a processor, implements the above-described device accessory detection method.

[0059] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0060] In the technical solution provided in this application, the edge computing node preloads a parts identification network, and uses the parts identification network to classify the acquired target parts images in the edge computing node. The classification results of the parts identification network are compared with the parts reference type information to detect parts, which improves the parts detection efficiency and reduces the false detection rate and false negative rate.

[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Furthermore, no embodiment in this application needs to achieve all the effects described above. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0063] Figure 1 This is a schematic flowchart illustrating an exemplary embodiment of the present application for a method of testing equipment parts;

[0064] Figure 2 This is a schematic flowchart illustrating an exemplary embodiment of this application of a method for detecting the body color of a vehicle.

[0065] Figure 3 This is a schematic diagram of the structure of an equipment accessory testing device shown in an exemplary embodiment of this application;

[0066] Figure 4 This is a hardware schematic diagram of an electronic device illustrated in an exemplary embodiment of this application. Detailed Implementation

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0068] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0069] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first classification threshold may also be referred to as a second classification threshold, and similarly, a second classification threshold may also be referred to as a first classification threshold. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0070] With changing market demands and product diversification, the variety of corresponding product components presents challenges to controlling errors and omissions in product assembly. For products or equipment that assemble multiple components, after the assembly process is completed, it is necessary to conduct component inspection to reduce quality defects such as misassembly and omissions, thereby improving product quality. For example, after the final assembly process of an automobile, it is necessary to perform matching inspections on the exterior components and some interior components of the entire vehicle.

[0071] Traditional equipment parts inspection processes are generally carried out manually. This method is inefficient and prone to omissions and errors when there are small differences between different models of equipment parts or when there are many types of equipment parts.

[0072] To address the aforementioned problems, this application provides a method for detecting equipment accessories. This method is applied to an edge computing node, which pre-loads an accessory identification network. See also... Figure 1 As shown, the method may include the following steps:

[0073] S101, acquire the image of the target accessory of the device under test;

[0074] The target accessory image is an image containing the accessory to be tested from the device under test. This image can be an image containing the target accessory that is directly acquired using an image acquisition device, or it can be an image containing the target accessory obtained by acquiring an image of the device under test using an image acquisition device and then processing the acquired image.

[0075] In one example, the device under test can be a device equipped with at least one accessory, and the accessory to be tested can be one or more accessories assembled in the device under test. The accessory type information of the one or more accessories can be identified through image recognition. For example, the device under test can be a vehicle, and the accessories to be tested can be exterior accessories, some interior accessories, etc. Exterior accessories include car logos, body colors, sunroofs, tire brands, door handles, wheel models, rearview mirrors, etc., while interior accessories include vanity mirrors, steering wheels, airbags, etc. The accessory type of these accessories can be determined through image recognition.

[0076] An image transmission connection is pre-established between the edge computing node and the image acquisition device. After the image acquisition device is triggered to start acquiring images, the edge computing node can obtain the image of the target accessory through the image transmission connection.

[0077] S102, the target accessory image is identified using the accessory recognition network to obtain the predicted classification result of the target accessory;

[0078] The accessory recognition network is pre-loaded into the edge computing nodes. This accessory recognition network is a trained neural network used to identify input images and predict their category.

[0079] After step S101 is completed, the edge computing node uses the acquired target component image as input to the component recognition network. The network then identifies the target component image to obtain a predicted classification result for the component. For example, if the edge computing node acquires a target component image of component A from the device under test, and component A corresponds to three types (A1, A2, and A3), the component recognition network predicts which of these three types (A1, A2, and A3) component A belongs to. If the target component image of component A is input into the component recognition network, and the network identifies the image and outputs a predicted classification result of A1, then the edge computing node obtains a predicted classification result of A1 for component A.

[0080] In one example, based on the detection of the vehicle's tire model, the edge computing node obtains the tire model image, which corresponds to three models: 215 / 60R1691H, 215 / 55R1690U, and 215 / 55R1685Q. The edge computing node uses the tire model image as input to the accessory recognition network. The accessory recognition network predicts that the tire model is the second of the three models, 215 / 55R1690U. Therefore, the edge computing node obtains the predicted classification result of the tire model as 215 / 55R1690U.

[0081] S103, compare the predicted classification result with the reference type information of the target accessory corresponding to the tested device to obtain the detection result of the target accessory.

[0082] The reference type information of the target accessory corresponding to the device under test refers to the type information of the target accessory that is theoretically installed on the device under test according to the device design requirements; the predicted classification result refers to the accessory type information obtained by the accessory identification network by identifying the image of the target accessory actually assembled on the device under test.

[0083] Taking a vehicle as an example, the vehicle has a parts list that indicates the reference type information of the target parts of the vehicle. Referring to Table 1, the table below shows a portion of the parts list for a certain series of vehicles. The information in the table indicates the reference type information of some parts in this series of vehicles. The parts shown in the table can have their type predicted by acquiring part images and using a part recognition network.

[0084]

[0085] Table 1. Partial List of Vehicles in a Certain Series

[0086] After step S102 is completed, the edge computing node, after obtaining the predicted classification result of the target accessory, compares the predicted classification result with the corresponding reference type information of the target accessory. If the predicted classification result and the reference type information are the same, the detection result of the target accessory is that the target accessory type is correctly matched; if the predicted classification result and the reference type information are different, the detection result of the target accessory is that the target accessory type is incorrectly matched.

[0087] Taking the tire model detection example again, the edge computing node obtains the predicted classification result of the tire model as 215 / 55R1690U. Assuming the tested device is the vehicle described in Table 1, the reference type information corresponding to the tire model is 215 / 60R1691H. The edge computing node compares the predicted classification result 215 / 55R1690U with the reference type information 215 / 60R1691H. The predicted classification result does not match the reference type information, resulting in the tire model detection result being "Tire model mismatch".

[0088] According to the technical solution provided in the embodiments of this application, the edge computing node preloads the accessory recognition network, and uses the accessory recognition network to classify the acquired target accessory images in the edge computing node. The classification results of the accessory recognition network are compared with the accessory reference type information to detect the accessory, which improves the accessory detection efficiency and reduces the false detection rate and false negative rate.

[0089] Furthermore, this detection method combines an accessory identification network with edge computing nodes, utilizing edge computing nodes to quickly respond to accessory detection and shortening the latency of detecting target accessories using related technologies.

[0090] In addition, edge computing nodes have lower costs than servers and have lower requirements for on-site equipment installation, so using edge computing nodes for anomaly detection can reduce costs.

[0091] In some embodiments, acquiring the target accessory image of the device under test may include: acquiring an image of the device under test; using a target detection network to detect the image to obtain an image region containing the target accessory; and obtaining an image of the target accessory based on the image region.

[0092] That is, the edge computing node first obtains an image from the image acquisition device of at least one part of the device under test, and the image contains at least one target accessory.

[0093] In one example, the device under test may be a vehicle, and images of the device under test may be acquired using multiple cameras positioned around the vehicle to acquire images of at least one part of the vehicle, including at least one target accessory of the vehicle, such as a tire accessory in an image of a vehicle part.

[0094] Next, an object detection network is used to perform object detection on the image. If the image contains a target accessory, the detection result includes the detection box of the target accessory, as well as the category and location information of the target accessory. Taking the vehicle part image including the tire obtained above as an example, the image is used as input to the object detection network. The object detection network finds the tire accessory and marks its position in the image with a detection box. For example, the object detection network detects the tire and outputs the position information of the detection box where the tire image area is located, the coordinates of the upper left corner of the detection box [x1, y1], and the coordinates of the lower right corner of the detection box [x2, y2].

[0095] Finally, based on the detection bounding boxes containing the target accessory output by the target detection network, an image containing the target accessory can be obtained. In other words, the image input to the target detection network can be cropped based on the detection bounding boxes to obtain the target accessory image. For example, based on the coordinates [x1, y1] and [x2, y2] of the two diagonal points of the detection bounding box containing the tire image region, the image of the vehicle part including the tire accessory can be cropped to obtain the tire image.

[0096] In this embodiment of the disclosure, a target detection network is used to perform target detection on the image of the device under test. The target component image is obtained based on the image region containing the target component output by the target detection network. This makes the obtained target component image contain more effective technical features, thereby improving the accuracy of target component image recognition and thus improving the accuracy of component detection.

[0097] In some embodiments, the edge computing node acquires images of target components of the device under test (DUT). Alternatively, it can directly acquire images of the target components using an image acquisition device, with a pre-established image transmission connection between the image acquisition device and the edge computing node. In one example, the image acquisition device may include at least one robot carrying a camera mounted on its cantilever arm, capable of transmitting images to the edge computing node. When the DUT reaches a preset testing station, the robot is triggered to move along a preset path to a designated image capture point near that station. The robot then moves the camera to the corresponding component location according to a preset component image capture order and acquires the component image, transmitting the captured image to the edge computing node. For example, if the DUT is a vehicle, the preset component image capture order is rearview mirror, door handle, and front tire. After reaching the first image capture point according to the preset path, the robot first moves the camera to the rearview mirror location to capture an image of the rearview mirror, and then sequentially moves to the door handle and front tire locations to capture images.

[0098] The image acquisition device may include two or more robots carrying cameras, and multiple designated shooting points can be preset. The robots can work simultaneously without interfering with each other, and each robot can acquire images of different accessories, thereby improving image acquisition efficiency.

[0099] Furthermore, after acquiring the image of the accessory, the corresponding image processing program can be triggered to further process the acquired accessory image, such as noise reduction, resolution enhancement, and background processing.

[0100] An accessory identification network is pre-loaded into the edge computing node. The accessory identification network is a trained neural network that can be constructed through transfer learning. The process mainly includes three steps: sample preparation, determining the network structure, and network training and verification.

[0101] First, a training sample set for the component recognition network is prepared based on the components to be inspected in the device under test. The component recognition network is used to identify images of the components to be inspected in the device under test. Based on the components to be inspected, an image acquisition device is used to acquire images of the corresponding components and label them with the corresponding type information, which serves as the training sample set. For example, if the device under test is a vehicle, a camera can be used to acquire images of car components and label their types, such as wheel hubs, door handles, rearview mirrors, tires, and body panels, and then label each type.

[0102] Among these features, images of components can be collected from different angles to enrich the training sample set; data augmentation operations can also be performed on the collected component images, such as random rotation, scaling, flipping, adding noise, and super-resolution enhancement using GAN networks, thereby expanding the training sample set and enhancing data quality.

[0103] Next, a pre-trained network is constructed using transfer learning to determine the network structure. This pre-trained accessory recognition network can be any lightweight classification network such as SqueezeNet, ShuffleNet, or MobileNet.

[0104] Then, the pre-trained network is trained using the training sample set. You can choose to freeze all layers of the pre-trained network except for the fully connected layers before training. During training, the network runs once on the training set and once on the validation set in each iteration, obtaining the corresponding loss and accuracy values ​​for the training and validation sets, respectively. Gradient descent is used to update the network parameters using the loss and accuracy values ​​from the training set. The generalization ability of the network is evaluated using the loss and accuracy values ​​from the validation set. After a certain number of training iterations, the loss value on the training set gradually decreases, while the loss value on the validation set gradually increases after reaching a minimum. The network corresponding to the minimum loss value on the training set can be used as the trained accessory recognition network.

[0105] In some embodiments, the step of using the accessory recognition network to identify the target accessory image and obtain the predicted classification result of the target accessory may include: obtaining the probability values ​​of the target accessory belonging to multiple preset categories; determining the maximum value among the multiple preset category probability values, and comparing the maximum value with a first classification threshold corresponding to the target accessory; if the maximum value is greater than or equal to the first classification threshold corresponding to the target accessory, determining the type corresponding to the maximum value as the predicted classification result of the target accessory.

[0106] Taking component A as an example, assuming component A includes three types: A1, A2, and A3, and the first classification threshold for component A in the component recognition network is 0.7. Using the component recognition network to identify the image of component A, the probability values ​​predicted by the network for component A to belong to A1, A2, and A3 are [0.76, 0.21, 0.03], respectively. The maximum probability value is determined to be 0.76. Since the maximum value of 0.76 is greater than the first classification threshold of 0.7, A1, corresponding to the maximum value of 0.76, is the predicted classification result for component A. The edge computing node then obtains the predicted classification result for component A as A1.

[0107] In some embodiments, the first classification threshold can be determined by the following steps: for the accessory identification network, obtain the recall and precision corresponding to the target accessory by setting different classification thresholds; determine the first classification threshold according to the precision-recall PR curve corresponding to the recall and precision.

[0108] Taking accessory A as an example again, for the trained accessory recognition network, based on the prediction results and training samples when the network is determined to be a trained accessory recognition network, several values ​​between (0,1) can be selected as classification thresholds. The recall and precision of the classification results for accessory A corresponding to these thresholds are then obtained. A precision-recall (PR) curve can be plotted, and the value corresponding to the point on the PR curve where recall and precision are equal is determined as the first classification threshold. Alternatively, the F1 value can be calculated using the following formula based on recall and precision, and an F1 value-classification threshold curve can be plotted. The classification threshold corresponding to the point with the largest F1 value is determined as the first classification threshold. The formula is:

[0109]

[0110] In this embodiment of the disclosure, the first classification threshold is determined by setting different classification thresholds to obtain the PR curve corresponding to the target accessory. The accessory recognition network with the first classification threshold has a better classification effect, thus improving the accuracy of the predicted classification result output by the accessory recognition network.

[0111] In some embodiments, after obtaining the detection results of the target accessory of the device under test, the edge computing node can also upload the target accessory image and the corresponding detection results to the cloud, so that the cloud can optimize the accessory recognition network; and update the accessory recognition network using the optimized accessory recognition network.

[0112] In other words, edge computing nodes can upload the target component image along with the corresponding component detection results to the cloud after obtaining the detection results of one or more components; alternatively, they can upload the results after obtaining the detection results of all components to be tested for the device under test. In one example, the device under test has a component list, which can be uploaded to the cloud together with the edge computing node after it starts detecting components or after all components to be tested for the device under test have been detected, in order to store the detection process data of the device under test.

[0113] In this process, after uploading the target accessory image and the corresponding target accessory image detection result to the cloud, the cloud can obtain reference type information for the target accessory. For target accessory images with a detection result of "correct match," the cloud can label the corresponding type information and use the labeled target accessory images as training samples. For example, if a tire image is obtained and the detection result is "tire model matched correctly," the cloud can obtain the tire's reference model information and label the tire image with the corresponding tire model.

[0114] The cloud uses training samples to optimize the parts recognition network, evaluates the performance of the optimized parts recognition network, and determines whether to update the parts recognition network. For example, by obtaining precision and recall to calculate the F1 value, if the F1 value is greater than the F1 value of the parts recognition network before optimization, the optimized parts recognition network is used to update the parts recognition network.

[0115] In this embodiment of the disclosure, the detected accessory images are uploaded to the cloud as new training samples to achieve iterative optimization of the accessory recognition network. The iterative optimization of the accessory recognition network improves the prediction accuracy of the accessory recognition network, thereby improving the accessory detection accuracy.

[0116] In some embodiments, for the optimized accessory recognition network, the cloud obtains the recall and precision corresponding to the target accessory by setting different classification thresholds; the edge computing node determines a second classification threshold based on the PR curves corresponding to the recall and precision, and updates the first classification threshold to the second classification threshold. The methods for obtaining the recall, precision, and determining the classification threshold in this embodiment are the same as those in the foregoing embodiments and will not be repeated here.

[0117] In some embodiments, the edge computing node may also send a prompt message in response to a discrepancy between the predicted classification result and the reference type information of the target accessory corresponding to the device under test.

[0118] For example, if the predicted classification result for accessory A of the tested equipment is A2, while the corresponding reference type information for accessory A is A1, then the predicted classification result and the reference type information comparison result for accessory A are inconsistent. In response to this inconsistency, the edge computing node can issue a warning to the testing personnel via an alarm device. This could include controlling an audible and visual alarm device to display a red light and provide a voice prompt stating "Accessory A type matching error." Alternatively, it can send a message to terminals, servers, displays, and other devices indicating that the predicted classification result and the reference type information comparison result for accessory A are inconsistent.

[0119] In this embodiment of the disclosure, a notification message is sent to remind staff to handle abnormalities, thereby reducing equipment quality defects and improving equipment quality.

[0120] The following section describes the proposed solution using specific application scenarios for equipment and component inspection. Taking automotive exterior component inspection as an example, after the final assembly process of automobile manufacturing and before performance testing, it is necessary to inspect automotive exterior components to prevent quality defects such as incorrect or missing components. These automotive exterior components may include body color, sunroof, glass type, tire type, car logo, and wheel model.

[0121] This application takes the inspection of an automotive exterior accessory as an example to illustrate the solution of this application. It is assumed that the wheel hub model is being inspected. The wheel hub model includes four types: C1, C2, C3, and C4. The wheel hub model in the design configuration list of the vehicle being tested is C1.

[0122] First, an accessory identification network is pre-loaded into the edge computing nodes. The accessory identification network has preset categories C1, C2, C3, and C4, corresponding to the order of probability values ​​for each preset category output by the network. The optimal classification threshold for wheel model in this accessory identification network is 0.85, meaning that the network performs best in classifying wheel models at this threshold, and the probability of correctly identifying the wheel model is highest at this threshold.

[0123] The following details the steps involved in detecting the wheel hub model; see [link to relevant documentation]. Figure 2 As shown, this step may include:

[0124] S201, acquire the wheel hub image of the vehicle under test;

[0125] A communication connection is pre-established between the edge computing node and the image acquisition device. The image acquisition device can be multiple industrial cameras placed around the vehicle under test. These industrial cameras can take pictures based on ambient light or automatically adjust their focus based on distance. Alternatively, the image acquisition device can be a camera mounted on a robot arm. When the robot moves the camera to a fixed shooting point, it triggers the camera to take an image of the accessory corresponding to that shooting point.

[0126] When the vehicle under test arrives at the designated detection position, the image acquisition device is triggered to start acquiring an image of the vehicle including the wheel hub and transmit the image to the edge computing node. After receiving the image, the edge computing node can further process the image, such as performing target detection to obtain the image region containing the wheel hub, and obtaining a more accurate wheel hub image based on the image region.

[0127] S202, the component recognition network is used to identify the wheel hub image to obtain a predicted classification result of the wheel hub image type;

[0128] The edge computing node uses the acquired wheel hub image as input to the parts recognition network, obtaining the probability value of the wheel hub belonging to a preset category from the network's output. For example, the output results are [0.88, 0.001, 0.11, 0.009], corresponding to wheel hub models [C1, C2, C3, C4], respectively. The maximum probability value is 0.88, corresponding to wheel hub model C1. Since the maximum value of 0.88 is greater than the optimal classification threshold of 0.85 for wheel hub models, wheel hub model C1 corresponding to this maximum value is determined as the preset classification result.

[0129] S203, compare the predicted classification result with the reference type information of the wheel hub model of the tested vehicle to obtain the detection result of the wheel hub model.

[0130] The predicted classification result of the wheel hub model is C1, and the reference type information of the wheel hub model of the tested vehicle is C1. It can be determined that the predicted classification result of the wheel hub model is consistent with the comparison result of the reference type information. Based on the consistency of the comparison result, the detection result of the wheel hub model is that the wheel hub model is correctly matched.

[0131] Furthermore, the edge computing node can control an alarm device to provide corresponding detection result prompts. This alarm device can be an audible and visual alarm. For example, if the edge computing node receives a detection result indicating that the wheel hub model is correctly matched, it can control the audible and visual alarm device to display a green light and provide a voice prompt stating "Wheel hub model matched correctly." Based on a similar principle, if the edge computing node receives a detection result indicating that the wheel hub model is incorrectly matched, it can control the audible and visual alarm device to display a red light and provide a voice prompt stating "Wheel hub model matched incorrectly."

[0132] This application provides a technical solution for applying a device parts detection method to automobile exterior inspection. An edge computing node preloads a parts identification network, which is used to classify the acquired target parts images in the edge computing node. The classification results of the parts identification network are compared with the parts reference type information to perform parts detection, thereby improving the parts detection efficiency and reducing the false detection rate and false negative rate.

[0133] Corresponding to the embodiments of the aforementioned equipment component testing methods, see [link to relevant documentation]. Figure 3 As shown, this application also provides an embodiment of an equipment accessory detection device, which is applied to an edge computing node, the edge computing node being pre-loaded with an accessory identification network; the device includes:

[0134] Parts image acquisition module 301 acquires images of target parts of the device under test;

[0135] The classification prediction module 302 uses the accessory recognition network to recognize the target accessory image and obtain the predicted classification result of the target accessory;

[0136] The information comparison module 303 compares the predicted classification result with the reference type information of the target accessory corresponding to the tested device to obtain the detection result of the target accessory.

[0137] In some embodiments, the accessory image acquisition module is specifically used to: acquire an image of the device under test; detect the image using a target detection network to obtain an image region containing the target accessory; and obtain an image of the target accessory based on the image region.

[0138] In some embodiments, the device under test includes a vehicle; acquiring images of the device under test may include: acquiring images of multiple parts of the vehicle using multiple cameras disposed around the vehicle.

[0139] In some embodiments, the classification prediction module is specifically used to: obtain the probability values ​​of the target accessory belonging to multiple preset categories; determine the maximum value among the probability values ​​of the multiple preset categories, and compare the maximum value with the first classification threshold corresponding to the target accessory; if the maximum value is greater than or equal to the first classification threshold corresponding to the target accessory, determine the type corresponding to the maximum value as the predicted classification result of the target accessory.

[0140] In some embodiments, the classification prediction module further includes: a first rate value acquisition module, configured to acquire the recall and precision corresponding to the target accessory by setting different classification thresholds for the accessory recognition network; and a threshold determination module, configured to determine the first classification threshold based on the precision-recall PR curve corresponding to the recall and precision.

[0141] In some embodiments, the apparatus further includes: an information transmission module, configured to upload the target accessory image and the corresponding detection result to the cloud when the detection result of the target accessory of the device under test is obtained, so that the cloud can optimize the accessory recognition network; and an identification network update module, configured to update the accessory recognition network using the optimized accessory recognition network.

[0142] In some embodiments, the identification network update module further includes: a second rate value acquisition module, used to acquire the recall and precision corresponding to the target accessory by setting different classification thresholds in the cloud for the optimized accessory identification network; and a threshold update module, used to determine a second classification threshold based on the PR curves corresponding to the recall and precision, and update the first classification threshold to the second classification threshold.

[0143] In some embodiments, the apparatus further includes a result response module, configured to send a prompt message in response to a discrepancy between the predicted classification result and the reference type information of the target accessory corresponding to the device under test.

[0144] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0145] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0146] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 4 As shown, the electronic device 400 includes at least one processor 401, a memory 402, and a bus 403. At least one processor 401 is electrically connected to the memory 402. The memory 402 is configured to store at least one computer-executable instruction, and the processor 401 is configured to execute the at least one computer-executable instruction to perform the steps of any device accessory detection method provided in any embodiment or optional embodiment of this application.

[0147] Furthermore, the processor 401 can be an FPGA (Field-Programmable Gate Array) or other devices with logic processing capabilities, such as an MCU (Microcontroller Unit) or a CPU (Central Processing Unit).

[0148] According to the technical solution provided in the embodiments of this application, the edge computing node preloads the accessory recognition network, and uses the accessory recognition network to classify the acquired target accessory images in the edge computing node. The classification results of the accessory recognition network are compared with the accessory reference type information to detect the accessory, which improves the accessory detection efficiency and reduces the false detection rate and false negative rate.

[0149] This application also provides another readable storage medium storing a computer program that, when executed by a processor, implements the steps of any device accessory testing method provided in any embodiment or optional implementation of this application.

[0150] The readable storage media provided in this application include, but are not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the readable storage media includes any medium by which a device (e.g., a computer) stores or transmits information in a readable form.

[0151] According to the technical solution provided in the embodiments of this application, the edge computing node preloads the accessory recognition network, and uses the accessory recognition network to classify the acquired target accessory images in the edge computing node. The classification results of the accessory recognition network are compared with the accessory reference type information to detect the accessory, which improves the accessory detection efficiency and reduces the false detection rate and false negative rate.

[0152] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0153] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0154] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0155] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for testing equipment parts, characterized in that, The method is applied to an edge computing node, which pre-loads an accessory identification network; the method includes: Acquire images of the target components of the device under test; The target component image is identified using the component identification network to obtain a predicted classification result for the target component; the predicted classification result refers to the component type information obtained by the component identification network from the image of the target component actually assembled in the tested equipment. The predicted classification result is compared with the reference type information of the target accessory corresponding to the device under test to obtain the detection result of the target accessory. The reference type information refers to the type information of the target accessory that is theoretically installed on the device under test according to the equipment design requirements. The step of comparing the predicted classification result with the reference type information of the target accessory corresponding to the tested device includes: obtaining the probability value of the target accessory belonging to multiple preset categories, and comparing the maximum value among the multiple preset categories with the first classification threshold corresponding to the target accessory; Upon obtaining the detection results of the target accessory of the device under test, the image of the target accessory and its corresponding detection results are uploaded to the cloud so that the cloud can optimize the accessory recognition network, evaluate the performance of the optimized accessory recognition network, and determine whether to update the accessory recognition network loaded on the edge computing node. Specifically, when the accessory recognition network is optimized, the cloud uses different classification thresholds to obtain the recall and precision corresponding to the target accessory. A second classification threshold is determined based on the PR curves corresponding to the recall and precision, and the first classification threshold corresponding to the target accessory is updated to the second classification threshold.

2. The method according to claim 1, characterized in that, The acquisition of the target component image of the device under test includes: Acquire an image of the device under test; The image is detected using a target detection network to obtain an image region containing the target accessory; The target accessory image is obtained based on the image region.

3. The method according to claim 1, characterized in that, The step of using the accessory recognition network to identify the target accessory image and obtain the predicted classification result of the target accessory includes: Obtain the probability values ​​of the target accessory belonging to multiple preset categories; The maximum value among the probability values ​​of the multiple preset categories is determined, and the maximum value is compared with the first classification threshold corresponding to the target accessory. If the maximum value is greater than or equal to the first classification threshold corresponding to the target accessory, the type corresponding to the maximum value is determined as the predicted classification result of the target accessory.

4. The method according to claim 3, characterized in that, The method further includes: For the accessory recognition network, the recall and precision corresponding to the target accessory are obtained by setting different classification thresholds; The first classification threshold is determined based on the precision-recall PR curve corresponding to the recall and precision.

5. A device for testing equipment parts, characterized in that, The device is applied to an edge computing node, which is pre-loaded with an accessory identification network; the device includes: The accessory image acquisition module acquires images of the target accessories of the device under test. The classification prediction module uses the accessory recognition network to identify the target accessory image and obtain the predicted classification result of the target accessory; the predicted classification result refers to the accessory type information obtained by the accessory recognition network from the image of the target accessory actually assembled in the tested equipment. The information comparison module compares the predicted classification result with the reference type information of the target accessory corresponding to the device under test to obtain the detection result of the target accessory. The reference type information refers to the type information of the target accessory that is theoretically installed on the device under test according to the equipment design requirements. The step of comparing the predicted classification result with the reference type information of the target accessory corresponding to the tested device includes: obtaining the probability value of the target accessory belonging to multiple preset categories, and comparing the maximum value among the multiple preset categories with the first classification threshold corresponding to the target accessory; The device further includes: upon obtaining the detection result of the target accessory of the device under test, uploading the target accessory image and its corresponding detection result to the cloud, so that the cloud can optimize the accessory recognition network, evaluate the performance of the optimized accessory recognition network, and determine whether to update the accessory recognition network loaded on the edge computing node; wherein, when the accessory recognition network is optimized, the cloud is used to obtain the recall and precision corresponding to the target accessory by setting different classification thresholds, a second classification threshold is determined according to the PR curve corresponding to the recall and precision, and the first classification threshold corresponding to the target accessory is updated to the second classification threshold.

6. A system for testing equipment parts, characterized in that, The system includes an image acquisition device and an edge computing node; the edge computing node is pre-loaded with an accessory recognition network. The image acquisition device is used to acquire images of multiple components of the device under test; The edge computing node is used to identify the target accessory image using the accessory recognition network to obtain the predicted classification result of the target accessory; The predicted classification result is compared with the reference type information of the target accessory corresponding to the device under test to obtain the detection result of the target accessory. The predicted classification result refers to the accessory type information obtained by the accessory recognition network from the image of the target accessory actually assembled in the device under test. The reference type information refers to the type information corresponding to the target accessory that is theoretically installed in the device under test according to the equipment design requirements. The step of comparing the predicted classification result with the reference type information of the target accessory corresponding to the tested device includes: obtaining the probability value of the target accessory belonging to multiple preset categories, and comparing the maximum value among the multiple preset categories with the first classification threshold corresponding to the target accessory; Upon obtaining the detection result of the target accessory of the device under test, the edge computing node uploads the target accessory image and its corresponding detection result to the cloud. This allows the cloud to optimize the accessory recognition network, evaluate the performance of the optimized network, and determine whether to update the accessory recognition network loaded on the edge computing node. Specifically, when the accessory recognition network is optimized, the cloud uses different classification thresholds to obtain the recall and precision corresponding to the target accessory. A second classification threshold is determined based on the PR curves corresponding to the recall and precision, and the first classification threshold corresponding to the target accessory is updated to the second classification threshold.

7. An electronic device, characterized in that, include: Processor, memory; The memory is used to store computer programs; The processor is configured to execute the equipment accessory testing method as described in any one of claims 1-4 by invoking the computer program.

8. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the equipment accessory detection method as described in any one of claims 1-4.

Images