Elbow plug-in mounting detection method and device for two devices of air conditioner
Through the machine learning-trained elbow insertion detection model, the elbow insertion image is processed, the matching score matrix and feature information is calculated, and the problem of automatic detection of elbow insertion in air conditioner manufacturing is solved, achieving efficient and accurate detection results.
Patent Information
- Application Number
- CN202411177822.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-26
AI Technical Summary
In the air conditioner manufacturing process, there are difficulties in automated detection of elbow insertion, existing methods are inefficient and costly, and due to the elbow shape and posture, it is difficult to achieve accurate detection.
The elbow insertion detection model obtained by machine learning training is adopted. By acquiring and processing the elbow insertion image, computing the matching score matrix, determining the target candidate area and feature information, and automatically detecting the elbow insertion.
Accurate and automated inspection of the elbow plug-in of the air conditioner two-device air conditioner has been realized, which reduces the difficulty of detection, improves production efficiency, reduces manual inspection requirements, and reduces production costs.
Smart Images

Figure CN119027404A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of automated production, and in particular to a method and a device for detecting elbow insertion of two devices of an air conditioner. Background Art
[0002] In the field of air-conditioning manufacturing, pipeline installation is a very important link, and elbow insertion is one of the key steps. Due to the complexity and diversity of incoming materials on the industrial production line, the shapes and sizes of the elbows are different, and the postures are changeable. Various defects are prone to occur during the insertion process, such as misalignment, tilt, and excessive gaps. These problems will affect product quality and production efficiency, so real-time monitoring and detection of elbow insertion is required. However, there is no mature elbow insertion detection method in the industry. The traditional elbow insertion detection method mainly relies on manual visual resolution, which is not only inefficient and error-prone, but also has high labor costs.
[0003] With the rapid development of industrial automation, the requirements for product quality and production efficiency are getting higher and higher, and more and more automated detection methods are often used in industrial production. However, the complex shape and changeable posture of elbow plugs make automated detection difficult.
[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0005] The embodiment of the present invention provides a method and device for detecting elbow insertion of two air conditioners, so as to at least solve the technical problem in the related art that elbow insertion is required in the production of air conditioners, but the automatic detection of elbow insertion is difficult due to the wide variety of elbows in the elbow insertion process.
[0006] According to one aspect of an embodiment of the present invention, a method for detecting elbow insertion of two air conditioners is provided, comprising: obtaining an elbow insertion image of two air conditioners, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow insertion image is an image of the two air conditioners with an elbow inserted; inputting the elbow insertion image into an elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: a sample elbow; An insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; determine the target candidate area in the elbow insertion image according to the matching score matrix, wherein the target candidate area includes each elbow; obtain feature information of each elbow in the target candidate area, wherein the feature information includes: position information of each elbow, angle information of each elbow; obtain the elbow insertion detection result of the two air conditioners according to the matching score matrix and the feature information.
[0007] Optionally, obtaining the elbow insertion image of the air conditioner includes: when it is determined that the air conditioner is transported to the elbow insertion area, controlling the material clamping component of the elbow insertion area to clamp the elbow and insert the elbow into the air conditioner; after determining that the elbow is inserted into the air conditioner, controlling the image acquisition device of the elbow insertion area to start, so as to acquire the elbow insertion image of the air conditioner.
[0008] Optionally, before inputting the elbow insertion image into the elbow insertion detection model to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, the elbow insertion detection method for the air conditioner further includes: acquiring the multiple sets of training data including the sample elbow insertion images and the sample matching score matrices; training the multiple sets of training data to obtain the elbow insertion detection model.
[0009] Optionally, the multiple sets of training data are trained to obtain the elbow insertion detection model, including: performing image preprocessing on the sample elbow insertion image and the template image to obtain the preprocessed sample elbow insertion image and the template image, wherein the template image is a top view image of the elbow; initializing the initial angle parameters, matching parameters and matching threshold of the elbow in the sample elbow insertion image, wherein the matching parameters are used to determine whether the sample elbow insertion image matches the template image, and the matching threshold is used to determine whether the template image is similar to the sample elbow image. The threshold score; randomly initialized to obtain a mask matrix of the same size as the sample elbow insertion image and the template image, so as to utilize The mask matrix performs filtering processing on the sample elbow insertion image and the template image to obtain the sample elbow insertion image and the template image after filtering; extracts the multi-scale features of the sample elbow insertion image after filtering; performs regional discrete weighted processing on the feature vectors at different positions of the sample elbow insertion image according to the multi-scale features to remove the feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow to obtain the retained feature vectors; performs sparse feature selection on the retained feature vectors to obtain sparse feature selection results; repeats the above steps until the error value between the obtained predicted matching score matrix and the sample matching score matrix is less than the error threshold, thereby obtaining the elbow insertion detection model.
[0010] Optionally, the elbow insertion detection method for two air conditioners also includes: setting the selection step and scaling factor of the template image; adjusting the angle and size of the template image according to the selection step and the scaling factor to obtain the adjusted template image; generating an elbow insertion template library according to the template image and the adjusted template image.
[0011] Optionally, the mask matrix includes a first mask matrix and a second mask matrix, which are randomly initialized to obtain a mask matrix of the same size as the sample elbow insertion image and the template image, so as to use the mask matrix to filter the sample elbow insertion image and the template image to obtain the filtered sample elbow insertion image and the template image, including: randomly initializing the first mask matrix and the second mask matrix composed of 0 and 1; using the first mask matrix to filter the template image to narrow the search area of the sample elbow insertion image; using the second mask matrix to filter the sample elbow insertion image to filter out the interference area in the sample elbow insertion image, wherein the interference area at least includes the background area of the sample elbow insertion image.
[0012] Optionally, extracting multi-scale features of the sample elbow insertion image after filtering includes: continuously downsampling and upsampling the sample elbow insertion image after filtering to obtain the sample elbow insertion images of different resolutions; performing feature extraction on the sample elbow insertion images of different resolutions to obtain the multi-scale features.
[0013] Optionally, feature vectors at different positions of the sample elbow insertion image are subjected to discrete weighted processing in different regions according to the multi-scale features to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, and obtain retained feature vectors, including: performing regional segmentation on the multi-scale features to obtain position information of each region in the sample elbow insertion image; assigning corresponding weights to the different positions according to the position information; and performing weighted summation processing on the different positions according to the weights to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, and obtain retained feature vectors.
[0014] Optionally, after obtaining the elbow insertion detection results of the two air conditioners according to the matching score matrix and the characteristic information, the elbow insertion detection method of the two air conditioners further includes: when the elbow insertion detection result indicates that there is a missing elbow in the two air conditioners, determining the coordinate information and angle information of the missing elbow according to the elbow insertion detection result; controlling the material clamping component to clamp the elbow according to the coordinate information and the angle information, and inserting the elbow to the missing elbow position corresponding to the coordinate information according to the angle information.
[0015] According to another aspect of an embodiment of the present invention, there is also provided an elbow insertion detection device for two air conditioners, comprising: a first acquisition module, used to obtain an elbow insertion image of two air conditioners, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow insertion image is an image of the two air conditioners with an elbow inserted; an input module, used to input the elbow insertion image into an elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: a sample elbow insertion image and A sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; a first determination module is used to determine the target candidate area in the elbow insertion image according to the matching score matrix, wherein the target candidate area includes each elbow; a second acquisition module is used to obtain feature information of each elbow in the target candidate area, wherein the feature information includes: position information of each elbow, angle information of each elbow; a third acquisition module is used to obtain the elbow insertion detection results of the two air conditioners according to the matching score matrix and the feature information.
[0016] Optionally, the first acquisition module includes: a first control unit, which is used to control the material clamping component of the elbow insertion area to clamp the elbow and insert the elbow into the air conditioner when it is determined that the air conditioner is transported to the elbow insertion area; and a second control unit, which is used to control the image acquisition device of the elbow insertion area to start after determining that the elbow is inserted into the air conditioner, so as to capture the elbow insertion image of the air conditioner.
[0017] Optionally, the elbow insertion detection device for the two air conditioners further includes: a fourth acquisition module, used to acquire the multiple groups of training data including the sample elbow insertion image and the sample matching score matrix before inputting the elbow insertion image into the elbow insertion detection model to process the elbow insertion image using the elbow insertion detection model to obtain the matching score matrix of the elbow insertion image; a training module, used to train the multiple groups of training data to obtain the elbow insertion detection model.
[0018] Optionally, the training module includes: a preprocessing unit, which is used to perform image preprocessing on the sample elbow insertion image and the template image to obtain the preprocessed sample elbow insertion image and the template image, wherein the template image is a top view image of the elbow; an initialization unit, which is used to initialize the initial angle parameters, matching parameters and matching thresholds of the elbow in the sample elbow insertion image, wherein the matching parameters are used to determine whether the sample elbow insertion image matches the template image, and the matching threshold is used to determine whether the template image is similar to the sample elbow image. The threshold score; a first processing unit, which is used to randomly initialize and obtain a mask matrix of the same size as the sample elbow insertion image and the template image, so as to use the mask matrix to initialize the sample elbow insertion image. The sample elbow insertion image and the template image are filtered to obtain the filtered sample elbow insertion image and the template image; a second processing unit is used to extract multi-scale features of the filtered sample elbow insertion image; a third processing unit is used to perform regional discrete weighted processing on feature vectors at different positions of the sample elbow insertion image according to the multi-scale features to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, so as to obtain retained feature vectors; a fourth processing unit is used to perform sparse feature selection on the retained feature vectors to obtain sparse feature selection results; a first acquisition unit is used to repeat the above steps until the error value between the obtained predicted matching score matrix and the sample matching score matrix is less than an error threshold, so as to obtain the elbow insertion detection model.
[0019] Optionally, the elbow insertion detection device for the two air conditioners further includes: a setting unit for setting the selection step and scaling factor of the template image; an adjustment unit for adjusting the angle and size of the template image according to the selection step and the scaling factor to obtain the adjusted template image; and a generation unit for generating an elbow insertion template library according to the template image and the adjusted template image.
[0020] Optionally, the first processing unit includes: a first processing sub-unit, used to randomly initialize the first mask matrix and the second mask matrix composed of 0 and 1; a second processing sub-unit, used to filter the template image using the first mask matrix to narrow the search area of the sample elbow insertion image; a third processing sub-unit, used to filter the sample elbow insertion image using the second mask matrix to filter out interference areas in the sample elbow insertion image, wherein the interference area at least includes the background area of the sample elbow insertion image.
[0021] Optionally, the second processing unit includes: a fourth processing sub-unit, used to continuously downsample and upsample the sample elbow insertion image after filtering to obtain the sample elbow insertion images with different resolutions; a feature extraction sub-unit, used to extract features from the sample elbow insertion images with different resolutions to obtain the multi-scale features.
[0022] Optionally, the third processing unit includes: a region segmentation subunit, used to perform region segmentation on the multi-scale features to obtain position information of each region in the sample elbow insertion image; an allocation subunit, used to assign corresponding weights to the different positions according to the position information; and a fifth processing subunit, used to perform weighted summation processing on the different positions according to the weights to remove feature vectors in the sample elbow insertion image that are not related to the geometric constraints of the elbow, and obtain retained feature vectors.
[0023] Optionally, the elbow insertion detection device for the two air conditioners further includes: a second determination module, for determining the coordinate information and angle information of the missing elbow according to the elbow insertion detection result after obtaining the elbow insertion detection result of the two air conditioners according to the matching score matrix and the characteristic information, when the elbow insertion detection result indicates that an elbow is missing in the two air conditioners; and a processing module, for controlling the material clamping component to clamp the elbow according to the coordinate information and the angle information, and inserting the elbow to the missing elbow position corresponding to the coordinate information according to the angle information.
[0024] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium comprising a stored program, wherein the program executes any one of the above-mentioned methods for detecting elbow insertion of two devices of an air conditioner.
[0025] According to another aspect of an embodiment of the present invention, a processor is further provided, the processor being used to run a program, wherein when the program is run, any one of the above-mentioned methods for detecting elbow insertion of two devices of an air conditioner is executed.
[0026] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, which, when executed by a processor, perform any one of the above-mentioned methods for detecting elbow insertion of two air conditioners.
[0027] In an embodiment of the present invention, an elbow insertion image of two air conditioners is obtained, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow insertion image is an image of the two air conditioners with an elbow inserted; the elbow insertion image is input into an elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: a sample elbow insertion image and a matching score matrix corresponding to the elbow insertion image. A sample matching score matrix corresponding to the sample elbow insertion image is obtained, where the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; a target candidate area in the elbow insertion image is determined according to the matching score matrix, wherein the target candidate area includes each elbow; characteristic information of each elbow in the target candidate area is obtained, wherein the characteristic information includes: position information of each elbow, angle information of each elbow; and elbow insertion detection results of the two air conditioners are obtained according to the matching score matrix and the characteristic information. Through the technical solution provided by the present invention, the acquired elbow insertion image of the two air conditioners is input into the pre-trained elbow insertion detection model to obtain the insertion score of the elbow in the elbow insertion image, and then the elbow with a higher insertion score is marked in the elbow insertion image, and the position and geometric constraints of the marked elbow are compared with the elbow with a higher score in the elbow insertion image to determine whether there is an elbow insertion abnormality in the two air conditioners, thereby achieving the technical effect of accurate automatic detection of the elbow insertion of the two air conditioners, reducing the difficulty of elbow insertion detection of the two air conditioners, and thus solving the technical problem in the related technology that elbow insertion is required in the production and manufacturing of air conditioners, but the automatic detection of elbow insertion is difficult due to the wide variety of elbows in the elbow insertion process. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0029] Figure 1 It is a hardware structure block diagram of a mobile terminal of a method for detecting elbow insertion of two air conditioners according to an embodiment of the present invention;
[0030] Figure 2 is a flow chart of a method for detecting elbow insertion of two devices of an air conditioner according to an embodiment of the present invention;
[0031] Figure 3 is a flow chart of the overall operation of the elbow insertion workstation according to an embodiment of the present invention;
[0032] Figure 4is a flow chart of an optional elbow plug-in detection method for two air conditioners according to an embodiment of the present invention;
[0033] FIG5( a ) is a three-dimensional schematic diagram of multi-scale discrete feature extraction according to an embodiment of the present invention;
[0034] FIG5( b ) is a two-dimensional schematic diagram of multi-scale discrete feature extraction according to an embodiment of the present invention;
[0035] Figure 6 Schematic diagram of an elbow plug-in detection device for two devices of an air conditioner according to an embodiment of the present invention. DETAILED DESCRIPTION
[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0038] As introduced in the background technology, in the related technology, elbow insertion is required in the production and manufacturing of air conditioners, and the technical problem that the automatic detection of elbow insertion is difficult due to the wide variety of elbows in the elbow insertion process. In view of the above defects, an elbow insertion detection method and device for air conditioners, a computer-readable storage medium, a processor and a computer program product are provided in the embodiments of the present invention.
[0039] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.
[0040] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal for detecting elbow plugging of two air conditioners according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0041] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the elbow plug-in detection method of the two air conditioners in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0042] According to an embodiment of the present invention, a method embodiment of a method for detecting elbow insertion of two air conditioners is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0043] Figure 2 FIG. 1 is a flow chart of a method for detecting elbow insertion of two air conditioners according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:
[0044] Step S202, obtaining an elbow-insertion image of two air conditioners, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow-insertion image is an image of the two air conditioners with elbows inserted.
[0045] In this embodiment, images of the evaporator and condenser of the air conditioner with elbows inserted can be obtained, which can be used as training data sets later to help the algorithm learn the elbow insertion features at different angles and scales.
[0046] Step S204, input the elbow insertion image into the elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model trained by machine learning using multiple sets of training data, each of the multiple sets of training data includes: a sample elbow insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to indicate whether each elbow is correctly plugged in.
[0047] In this embodiment, the elbow insertion image can be input into the elbow insertion detection model obtained by machine learning training based on multiple sets of training data. By using the elbow insertion model to process the elbow insertion image to obtain a matching score matrix of the elbow insertion image, it can help determine whether each elbow is correctly plugged in and promptly discover and correct insertion errors.
[0048] Figure 3 FIG. 1 is a flow chart of the overall operation of the elbow insertion workstation according to an embodiment of the present invention. Figure 3 As shown, the elbow insertion image is input into the elbow insertion detection model, and the elbow insertion detection is performed to determine whether there is a missed insertion. Here, the judgment can be made based on the above matching score matrix.
[0049] Step S206, determining a target candidate region in the elbow insertion image according to the matching score matrix, wherein the target candidate region includes each elbow.
[0050] In this embodiment, each elbow in the elbow insertion image can be determined according to the matching score matrix to accurately locate and identify the position of the elbow in the image, thereby improving the accuracy of target detection.
[0051] Step S208, acquiring feature information of each elbow in the target candidate area, wherein the feature information includes: position information of each elbow and angle information of each elbow.
[0052] In this embodiment, obtaining feature information of the elbow in the target candidate area can help achieve accurate positioning, identification, tracking and analysis of the target, and provide a basis for subsequent decision-making and processing.
[0053] Step S210, obtaining elbow insertion detection results of the two air conditioners according to the matching score matrix and the feature information.
[0054] It should be noted that the position coordinate angle and score of the inserted elbow in the result set output by the above-mentioned elbow insertion detection model are compared with the position coordinates (pre-set) of all elbows that need to be inserted in the current detection image. Within a certain deviation range, the two are considered to be corresponding positions. At this time, the confidence score of the elbow is obtained. If it is less than the matching threshold, it is considered that there is a missed elbow.
[0055] In this embodiment, the elbow plug-in detection results of the two air conditioners can be obtained based on the comprehensive analysis of the matching score matrix and the feature information. If the matching score is high and the position information and angle information are consistent with expectations, the detection result is determined to be correct, and automatic detection and monitoring can be achieved to improve production efficiency. That is, after multiple iterative matching, the best matching score matrix can be generated (the error function is the minimum at this time). According to the matching score matrix, the highest-scoring feature point pairs and their corresponding areas can be extracted, which are the candidate target areas. The position information (i.e., center point coordinates, width and height), elbow angle, and confidence score of each elbow candidate area are written into the elbow plug-in detection result set. That is, during the plug-in process, the elbow may have a certain offset in angle and position, so the elbow can be matched again through the angle, position coordinates and other information of the entire image to improve the accuracy of the elbow plug-in detection of the two devices.
[0056] As can be seen from the above, in an embodiment of the present invention, an elbow insertion image of two air conditioners is obtained, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow insertion image is an image of the two air conditioners with elbows inserted; the elbow insertion image is input into an elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: a sample elbow insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; the target candidate area in the elbow insertion image is determined according to the matching score matrix domain, wherein the target candidate area includes each elbow; characteristic information of each elbow in the target candidate area is obtained, wherein the characteristic information includes: position information of each elbow and angle information of each elbow; the elbow insertion detection result of the two air conditioners is obtained according to the matching score matrix and the characteristic information, so as to input the acquired elbow insertion image of the two air conditioners into the pre-trained elbow insertion detection model to obtain the insertion score of the elbow in the elbow insertion image, and then mark the elbow with a higher insertion score in the elbow insertion image, and compare the position and geometric constraints of the marked elbow with the elbow with a higher score in the elbow insertion image to determine whether there is elbow insertion abnormality in the two air conditioners, thereby achieving the technical effect of accurate automatic detection of elbow insertion of the two air conditioners, and reducing the difficulty of elbow insertion detection of the two air conditioners.
[0057] The technical solution provided by the embodiment of the present invention solves the technical problem in the related art that elbow insertion is required in the production of air conditioners, but the automatic detection of elbow insertion is difficult due to the wide variety of elbows in the elbow insertion process.
[0058] According to the above embodiment of the present invention, obtaining the elbow insertion image of the two air conditioners includes: when it is determined that the two air conditioners are transported to the elbow insertion area, controlling the material clamping components in the elbow insertion area to clamp the elbow and insert the elbow into the two air conditioners; after determining that the elbow is inserted into the two air conditioners, controlling the image acquisition device in the elbow insertion area to start, so as to acquire the elbow insertion image of the two air conditioners.
[0059] like Figure 3As shown, the materials of the two devices of the air conditioner after the previous process (i.e., inserting the elbows into the two devices) are automatically loaded to the elbow insertion workstation via the assembly line, the elbow material loading tray rotates, and the robot (i.e., the material clamping component) clamps the elbow material to prepare for the next operation, and inserts the elbow into the two devices of the air conditioner. After confirming that the elbow is inserted into the two devices of the air conditioner, the camera in the elbow insertion area (i.e., the image acquisition device) is controlled to take pictures of the two device materials to capture the source image (i.e., the elbow insertion image of the two devices of the air conditioner).
[0060] According to the above-mentioned embodiment of the present invention, before the elbow insertion image is input into the elbow insertion detection model to process the elbow insertion image using the elbow insertion detection model to obtain the matching score matrix of the elbow insertion image, the elbow insertion detection method of the air conditioner further includes: acquiring multiple groups of training data including sample elbow insertion images and sample matching score matrices; training the multiple groups of training data to obtain the elbow insertion detection model.
[0061] In this embodiment, the above-mentioned sample elbow insertion images and the sample matching score matrix corresponding to each sample elbow insertion image can be obtained in advance, so that the above-mentioned elbow insertion detection model can be trained using the sample elbow insertion images and the sample matching score matrix corresponding to each sample elbow insertion image, so that in the subsequent elbow insertion detection of the two air-conditioning devices, the collected elbow insertion images of the two air-conditioning devices can be input into the elbow insertion detection model, and the preliminary result of the elbow insertion can be obtained through the elbow insertion detection model, thereby improving the efficiency of the elbow insertion detection of the two air-conditioning devices.
[0062] According to the above embodiment of the present invention, multiple sets of training data are trained to obtain an elbow insertion detection model, including: performing image preprocessing on a sample elbow insertion image and a template image to obtain a preprocessed sample elbow insertion image and a template image, wherein the template image is a top view image of the elbow; initializing the initial angle parameter, matching parameter and matching threshold of the elbow in the sample elbow insertion image, wherein the matching parameter is used to determine whether the sample elbow insertion image matches the template image, and the matching threshold is used to determine whether the template image is similar to the sample elbow image. The threshold score; randomly initialized to obtain a mask matrix of the same size as the sample elbow insertion image and the template image, so as to utilize The mask matrix is used to filter the sample elbow insertion image and the template image to obtain the filtered sample elbow insertion image and the template image; the multi-scale features of the filtered sample elbow insertion image are extracted; the feature vectors at different positions of the sample elbow insertion image are subjected to regional discrete weighting processing according to the multi-scale features to remove the feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow to obtain the retained feature vectors; sparse feature selection is performed on the retained feature vectors to obtain the sparse feature selection results; the above steps are repeated until the error value between the obtained prediction matching score matrix and the sample matching score matrix is less than the error threshold to obtain the elbow insertion detection model.
[0063] Optionally, the template image is a top view image of a marked elbow, including some geometric constraint information of the elbow, such as the elbow geometry, the elbow center distance, and the elbow diameter.
[0064] Figure 4 FIG. 1 is a flow chart of an optional elbow plug-in detection method for two air conditioners according to an embodiment of the present invention. Figure 4 As shown, in this embodiment, the sample elbow insertion image and the template image can be preprocessed first, for example, (1) grayscale processing: converting the input image into a grayscale image to reduce the complexity of subsequent algorithm calculations; (2) grayscale image scaling processing, adjusting the grayscale range of the image through nonlinear transformation; (3) filtering operation. Use an image filter (such as a Gaussian image filter) to reduce interference such as clutter and noise in the image. The image preprocessing operation here can remove the interference of impurities and some noise, which is convenient for subsequent image analysis and processing.
[0065] Next, you can initialize the angle parameters, matching parameters, and set the matching threshold. Initializing the angle parameters means giving the elbow an initial angle range, which can be set to 0-360 degrees. To reduce the search time, it can be set to 0°. The matching parameters are features (edges, contours, etc.) used to determine whether the target image matches the elbow image. The matching threshold set here is the threshold score used to determine whether the elbow template image is similar enough to the target image and is considered a match.
[0066] In addition, two mask matrices (i.e., mask matrices) can be randomly initialized, with the same size as the template image and the target image to be matched (i.e., the sample elbow insertion image in the model training phase), where 0 represents the masked part and 1 represents the unmasked part, and element-by-element dot product is performed with the input image data. This step uses the mask matrix to control which pixels participate in the calculation and subsequent processing.
[0067] FIG5(a) is a three-dimensional schematic diagram of multi-scale discrete feature extraction according to an embodiment of the present invention. As shown in FIG5(a), a feature pyramid can be used to extract multi-scale features, and feature vectors at different positions are weighted by regions, and irrelevant features outside the geometric constraints of the elbow are removed, and only feature vectors with higher weights are retained. For example, first, a feature pyramid is constructed to obtain feature maps of different scales; second, the sizes of feature maps of different scales are adjusted by upsampling and downsampling, and feature maps of different scales are fused to generate a unified multi-scale feature representation; then, the fused multi-scale feature map is segmented, and the corresponding weights are assigned to each region according to the position of each region in the image and according to prior knowledge (i.e., the position of the elbow ROI region), so as to reduce the interference of the background region; at the same time, the target region is further reduced according to the geometric constraints of the elbow (i.e., the angle, contour, size, etc. of the template elbow); finally, the feature vectors at different positions are weighted and summed to obtain the weighted feature vector of each region, and in the process of multiple iterative matching, the weights are dynamically adjusted to remove the feature vectors with lower weights and retain the feature vectors with higher weights.
[0068] In this embodiment, as shown in FIG. 5( a ), the idea of discretization search is also used to perform sparse feature selection on the feature vector, that is, the weight factor α of the L1 norm regularization is adjusted, and the value range of α is discretized into a finite number of values α={α 1 ,α 2 ,...,α n}, and select the optimal α value as the final sparse feature selection result. In this step of sparse feature selection, the goal is to find the optimal weight factor α, so that while ensuring a certain performance, the number of feature vectors can be reduced as much as possible, thereby reducing model complexity and computation. L1 regularization adds an L1 norm term, which is the sum of the absolute values of feature coefficients. Using L1 regularization in the process of multiple iterative matching can make some feature coefficients become 0. If the feature coefficient is 0, the feature vector is considered unimportant and can be removed without subsequent processing, thereby achieving the purpose of sparse selection.
[0069] The above steps are iterated and matched multiple times. In the i-th iteration, the matching matrix M(i) containing all the matching scores contains the feature information of those points with corresponding relationships. Two subsets are generated based on the matching matrix. One subset constrains the geometric features, calculates the feature representation of the current constraint area, removes the matching pairs outside the constraints, and the other subset retains the original key feature information. At the same time, the matching points within the geometric constraint range are used to enhance the key feature information. In each iteration, this process is repeated, and the parameters are continuously adjusted to generate the best matching score matrix to minimize the error function. The error function calculation formula is: This formula calculates the error function between the subset x with set feature constraints and the subset y with key feature information, that is, it describes the similarity and difference between them in elbow features, where i represents the i-th iteration, j represents the j-th feature vector, mn represents the size of the subset (i.e., feature dimension), and p l Represents the weight factor vector of the key features (which can describe the importance of the lth key feature vector in the distance calculation when calculating the distance between the two subsets), is a subset sample with geometric feature constraints, is a subset sample with key feature information, diff(x,y) represents the distance function between two subsets, p j is the weight factor.
[0070] The key feature information is the feature vector after feature vector discretization weighting and sparse feature selection, which is the weighted and screened feature (excluding background, irrelevant areas, noise interference, etc.), that is, the key feature representation of the elbow area. It includes the contour shape of the elbow, the rotation angle of the elbow, and the position relationship information.
[0071] Among them, the above-mentioned use of matching points within the geometric constraint range to enhance key feature information can first determine the geometric constraint conditions of the elbow (the angle position size of the elbow, the contour shape of the elbow, the position restriction relationship of each edge point of the elbow, etc.), and the geometric constraint range here is the area within the outer edge contour of the elbow (the area outside the elbow contour is not processed). Then the original features of the current pixel matching point are fused with the key feature information obtained after a series of processing.
[0072] According to the above embodiment of the present invention, the elbow insertion detection method for the two air conditioners also includes: setting the selection step and scaling factor of the template image; adjusting the angle and size of the template image according to the selection step and scaling factor to obtain the adjusted template image; generating an elbow insertion template library according to the template image and the adjusted template image.
[0073] In this embodiment, based on the template image, a template library containing elbows of different angles and sizes can be created, and parameters such as selection step size and scaling factor can be set. The purpose of setting the step size is to provide a fine search of all angles for building a multi-angle elbow template library. After initializing an angle value, a selection step size needs to be set, such as 2°, which can be understood as performing a search every 2° rotation. The scaling factor is used to adjust the size of the elbow template image to adapt to elbow images of different scales, and is generally set to 0.8, 1, 1.2, or 1.4.
[0074] In the above embodiment of the present invention, the mask matrix includes a first mask matrix and a second mask matrix, and the mask matrix of the same size as the sample elbow insertion image and the template image is randomly initialized to filter the sample elbow insertion image and the template image using the mask matrix to obtain the filtered sample elbow insertion image and the template image, including: randomly initializing the first mask matrix and the second mask matrix composed of 0 and 1; using the first mask matrix to filter the template image to narrow the search area of the sample elbow insertion image; using the second mask matrix to filter the sample elbow insertion image to filter out the interference area in the sample elbow insertion image, wherein the interference area at least includes the background area of the sample elbow insertion image.
[0075] The mask matrix is a binary matrix. 0 and 1 respectively indicate whether the pixel in the image is masked (i.e., whether it participates in the calculation). In the subsequent dot product operation, if the pixel is 0, the pixels of the template image and the target image to be matched corresponding to these pixels are ignored and do not participate in the subsequent processing. The positions of 0 and 1 in the two mask matrices are randomly generated at this time, and will be continuously updated and changed in the subsequent multiple iterative matching processes. The functions of the two mask matrices are: one is the mask matrix of the template matching image, which is used to describe the key features of the elbow in the template image and narrow the search area in the image to be matched. The other is the mask matrix of the target image to be matched, which is used to exclude most of the interference areas, such as noise or cluttered background (set this part to zero). Through this part, the image can be further filtered to improve the subsequent image processing speed.
[0076] In the above embodiment of the present invention, multi-scale features of the sample elbow insertion image after filtering are extracted, including: continuously downsampling and upsampling the sample elbow insertion image after filtering to obtain sample elbow insertion images of different resolutions; and extracting features from the sample elbow insertion images of different resolutions to obtain multi-scale features.
[0077] In the above embodiment of the present invention, feature vectors at different positions of the sample elbow insertion image are subjected to discrete weighted processing in different regions according to multi-scale features to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, and obtain retained feature vectors, including: performing regional segmentation on the multi-scale features to obtain position information of each region in the sample elbow insertion image; assigning corresponding weights to different positions according to the position information; and performing weighted summation processing on different positions according to the weights to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, and obtain retained feature vectors.
[0078] The multi-scale features extracted by the feature pyramid here include the contour shape of the elbow, the rotation angle of the elbow, the position coordinates of the elbow in the image and the relative position relationship, etc. Among them, the feature vector representation of different positions of the above elbow is generated according to the feature pyramid. The core idea is to gradually reduce the image resolution by continuously downsampling the original size image, that is, decomposing the original size image into a series of image sets of different scales and then extracting its feature representation. Figure 5 (b) is a two-dimensional schematic diagram of multi-scale discrete feature extraction according to an embodiment of the present invention. As shown in Figure 5 (b), the two-device elbow insertion image is sampled to obtain three layers of images, and each layer represents the detailed feature information of different scales. Therefore, the feature vector extracted here contains feature information of multiple scales. The structural features and edge information of the elbow can be extracted from the high-resolution feature map, and the context information in a larger range can be extracted from the low-resolution feature map (that is, the position relationship of the target elbow position in the entire image and the relative position relationship with other elbows, etc.).
[0079] According to the above embodiment of the present invention, after obtaining the elbow insertion detection results of the two air conditioners according to the matching score matrix and the feature information, the elbow insertion detection method of the two air conditioners further includes: when the elbow insertion detection results indicate that there is a missing elbow in the two air conditioners, determining the coordinate information and angle information of the missing elbow according to the elbow insertion detection results; controlling the material clamping component to clamp the elbow according to the coordinate information and the angle information, and inserting the elbow to the missing elbow position corresponding to the coordinate information according to the angle information.
[0080] In this embodiment, if Figure 3 As shown, the elbow insertion detection algorithm processes the ROI area according to the image data collected by photography, detects the elbow insertion status in the ROI area, and outputs a result set; the elbow insertion detection algorithm can also determine whether there is a missed elbow according to the output result set. If so, the positioning coordinates and angles of the missed elbow are output and converted into robot coordinates; then the robot implements the insertion operation of the missed elbow according to the algorithm output results; finally, the two devices and the elbow material flow out.
[0081] Here, the result set output by the elbow insertion detection algorithm contains the position coordinate angle and score of the installed elbow. The position coordinates (pre-set) of all elbows that need to be inserted in the current detection image are compared with them. If the deviation is within a certain range, the two are considered to be corresponding positions. At this time, the confidence score of the elbow is obtained. If it is less than the matching threshold, it is considered that there is a missed elbow.
[0082] Through the technical solution provided by the above-mentioned embodiment of the present invention, a matching algorithm based on multi-scale discrete features is first proposed, the search space of the elbow target is discretized using mask technology, an image pyramid is constructed to extract multi-scale features, and sparse feature selection is performed in combination with the key position of the elbow, thereby optimizing the feature matching process of the algorithm and improving the matching efficiency of the algorithm, thereby significantly improving the accuracy and processing speed of elbow insertion detection. In addition, a multi-angle iterative matching algorithm with adaptive geometric constraints is proposed, which adaptively adjusts the elbow template image to construct a rich multi-angle template library, performs precise geometric constraints on the matching process according to the size and shape characteristics of the elbow, determines the angle deviation range during the insertion process, and finely adjusts and optimizes the angle parameters and matching parameters of the elbow through multiple iterative matching, thereby improving the detection accuracy and robustness of the algorithm in complex environments.
[0083] In the technical solution provided in the above embodiment of the present invention, in view of the problems of dislocation, tilt, and excessive gap caused by the complex incoming materials of the assembly line in the industrial production scene, many interference factors, and the changeable shapes and postures of the elbows, and the high positioning accuracy and fast beat requirements during elbow insertion detection, the present invention proposes the following two innovations: first, multi-scale discrete feature matching and key position sparse feature selection are proposed, so that the algorithm can extract elbow features at different scales, so that it only focuses on the significant features of the elbow target, reduces the algorithm's dependence on noise and irrelevant features, thereby reducing the false detection rate and unnecessary calculation amount, and improving the detection efficiency. Second, adaptive geometric constraints and multi-angle iterative matching are proposed, so that the algorithm can cover the appearance of the elbow target at different angles, adapt to changes in different lighting, occlusion and background, reduce detection failures caused by environmental changes, and thus improve the robustness of the algorithm. In summary, the elbow insertion detection method proposed by the present invention can realize high-precision and high-efficiency elbow insertion detection at the industrial production site, thereby reducing the need for manual detection, reducing production costs and improving the automation level of the production line. In addition, the application of the present invention can improve product quality and reduce product defects caused by elbow insertion errors. At the same time, in view of the complex manufacturing background and the interference of factors such as illumination changes in the on-site production environment, and the unknown size and position of the elbow target, the search space and algorithm calculation amount are extremely large. By proposing a matching algorithm based on multi-scale discrete features, the feature representation of the elbow target can be accurately extracted, the search space can be greatly reduced, and the rapid and accurate positioning of the elbow target under the complex background can be achieved; in view of the angle deviation of the two devices in the on-site production environment, there is a certain rotation angle after the elbow insertion is completed, resulting in a large number of mismatches in the existing elbow template image matching. By proposing a multi-angle iterative matching algorithm with adaptive geometric constraints, the matching accuracy of the algorithm can be improved, and the multi-angle matching of the elbow target can be achieved while ensuring that the template image information is not lost. In addition, the quality and efficiency of elbow insertion are also improved, and the industrial production cost is reduced.
[0084] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0085] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.
[0086] According to an embodiment of the present invention, there is also provided an elbow plug-in detection device for an air conditioner for implementing the above-mentioned elbow plug-in detection method for an air conditioner. Figure 6 Schematic diagram of an elbow plug-in detection device for two air conditioners according to an embodiment of the present invention. Figure 6 As shown, the device includes: a first acquisition module 601, an input module 603, a first determination module 605, a second acquisition module 607 and a third acquisition module 609. The risk analysis device in the operation process of the power grid is described below.
[0087] The first acquisition module 601 is used to acquire an elbow-insertion image of two air conditioners, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow-insertion image is an image of the two air conditioners with the elbow inserted;
[0088] An input module 603 is used to input the elbow insertion image into the elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample elbow insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in;
[0089] A first determination module 605 is used to determine a target candidate region in the elbow insertion image according to a matching score matrix, wherein the target candidate region includes each elbow;
[0090] The second acquisition module 607 is used to acquire feature information of each elbow in the target candidate area, wherein the feature information includes: position information of each elbow and angle information of each elbow;
[0091] The third acquisition module 609 is used to obtain the elbow insertion detection results of the two air conditioners according to the matching score matrix and the feature information.
[0092] It should be noted here that the above-mentioned first acquisition module 601, input module 603, first determination module 605, second acquisition module 607 and third acquisition module 609 correspond to steps S202 to S210 in the above-mentioned embodiments, and the five modules are the same as the instances and application scenarios implemented by the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiments.
[0093] As can be seen from the above, in the scheme recorded in the above-mentioned embodiment of the present invention, first, the first acquisition module can be used to acquire the elbow insertion image of the two air conditioners, wherein the two air conditioners include: the evaporator and the condenser of the air conditioner, and the elbow insertion image is an image of the two air conditioners with elbows inserted; then the input module can be used to input the elbow insertion image into the elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data includes: a sample elbow insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; then the first determination module can be used to determine the elbow insertion image according to the matching score matrix. a target candidate area in the target candidate area, wherein the target candidate area includes each elbow; secondly, the feature information of each elbow in the target candidate area can be obtained by using the second acquisition module, wherein the feature information includes: position information of each elbow and angle information of each elbow; finally, the elbow insertion detection result of the two air conditioners can be obtained by using the third acquisition module according to the matching score matrix and the feature information, so as to input the acquired elbow insertion image of the two air conditioners into the pre-trained elbow insertion detection model to obtain the insertion score of the elbow in the elbow insertion image, and then mark the elbow with a higher insertion score in the elbow insertion image, and compare the position and geometric constraints of the marked elbow with the elbow with a higher score in the elbow insertion image to determine whether there is an elbow insertion abnormality in the two air conditioners, thereby achieving the technical effect of accurate and automatic detection of the elbow insertion of the two air conditioners, and reducing the difficulty of elbow insertion detection of the two air conditioners.
[0094] The technical solution provided by the embodiment of the present invention solves the technical problem in the related art that elbow insertion is required in the production of air conditioners, but the automatic detection of elbow insertion is difficult due to the wide variety of elbows in the elbow insertion process.
[0095] In an optional embodiment, the first acquisition module includes: a first control unit, which is used to control the material clamping component in the elbow insertion area to clamp the elbow and insert the elbow into the air conditioner when it is determined that the air conditioner is transported to the elbow insertion area; a second control unit, which is used to control the image acquisition device in the elbow insertion area to start after determining that the elbow is inserted into the air conditioner, so as to collect the elbow insertion image of the air conditioner.
[0096] In an optional embodiment, the elbow insertion detection device for the two air conditioners further includes: a fourth acquisition module, which is used to obtain multiple groups of training data including sample elbow insertion images and sample matching score matrices before inputting the elbow insertion image into the elbow insertion detection model to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image; and a training module, which is used to train the multiple groups of training data to obtain the elbow insertion detection model.
[0097] In an optional embodiment, the training module includes: a preprocessing unit, which is used to perform image preprocessing on the sample elbow insertion image and the template image to obtain the preprocessed sample elbow insertion image and the template image, wherein the template image is a top view image of the elbow; an initialization unit, which is used to initialize the initial angle parameters, matching parameters and matching thresholds of the elbow in the sample elbow insertion image, wherein the matching parameters are used to determine whether the sample elbow insertion image and the template image match the features, and the matching threshold is used to determine whether the template image and the sample elbow image are similar. The threshold score; a first processing unit, which is used to randomly initialize and obtain a mask matrix of the same size as the sample elbow insertion image and the template image, so as to use the mask matrix to initialize the sample elbow insertion image and The template image is filtered to obtain a sample elbow insertion image and a template image after filtering; a second processing unit is used to extract multi-scale features of the sample elbow insertion image after filtering; a third processing unit is used to perform regional discrete weighted processing on feature vectors at different positions of the sample elbow insertion image according to the multi-scale features to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, and obtain retained feature vectors; a fourth processing unit is used to perform sparse feature selection on the retained feature vectors to obtain sparse feature selection results; a first acquisition unit is used to repeat the above steps until the error value between the obtained predicted matching score matrix and the sample matching score matrix is less than the error threshold, and the elbow insertion detection model is obtained.
[0098] In an optional embodiment, the elbow insertion detection device for the two air conditioners also includes: a setting unit, used to set the selection step and scaling factor of the template image; an adjustment unit, used to adjust the angle and size of the template image according to the selection step and scaling factor to obtain the adjusted template image; a generation unit, used to generate an elbow insertion template library according to the template image and the adjusted template image.
[0099] In an optional embodiment, the first processing unit includes: a first processing sub-unit, used to randomly initialize a first mask matrix and a second mask matrix composed of 0 and 1; a second processing sub-unit, used to filter the template image using the first mask matrix to narrow the search area of the sample elbow insertion image; a third processing sub-unit, used to filter the sample elbow insertion image using the second mask matrix to filter out the interference area in the sample elbow insertion image, wherein the interference area at least includes the background area of the sample elbow insertion image.
[0100] In an optional embodiment, the second processing unit includes: a fourth processing sub-unit, used to continuously downsample and upsample the sample elbow insertion image after filtering to obtain sample elbow insertion images of different resolutions; a feature extraction sub-unit, used to extract features from the sample elbow insertion images of different resolutions to obtain multi-scale features.
[0101] In an optional embodiment, the third processing unit includes: a region segmentation subunit, which is used to perform region segmentation on the multi-scale features to obtain the position information of each region in the sample elbow insertion image; an allocation subunit, which is used to assign corresponding weights to different positions according to the position information; and a fifth processing subunit, which is used to perform weighted summation processing on different positions according to the weights to remove feature vectors in the sample elbow insertion image that are not related to the geometric constraints of the elbow, and obtain retained feature vectors.
[0102] In an optional embodiment, the elbow insertion detection device for the two air conditioners further includes: a second determination module, for determining the coordinate information and angle information of the missing elbow according to the elbow insertion detection result after obtaining the elbow insertion detection result of the two air conditioners according to the matching score matrix and the characteristic information, when the elbow insertion detection result indicates that there is a missing elbow in the two air conditioners; a processing module, for controlling the material clamping component to clamp the elbow according to the coordinate information and the angle information, and inserting the elbow to the missing elbow position corresponding to the coordinate information according to the angle information.
[0103] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein when the program is run, any one of the above-mentioned methods for detecting elbow insertion of two devices of an air conditioner is executed.
[0104] According to another aspect of an embodiment of the present invention, a computer program product is provided, comprising computer instructions, which, when executed by a processor, execute any one of the above-mentioned methods for detecting elbow installation of two air conditioners.
[0105] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium comprising a stored program, wherein the program executes any one of the above-mentioned methods for detecting elbow insertion of two devices of an air conditioner.
[0106] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the communication devices in a communication device group.
[0107] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: obtaining an elbow-insertion image of two air conditioners, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow-insertion image is an image of the two air conditioners with elbows inserted; inputting the elbow-insertion image into an elbow-insertion detection model, so as to process the elbow-insertion image using the elbow-insertion detection model to obtain a matching score matrix of the elbow-insertion image, wherein the elbow-insertion detection model is a model obtained by machine learning training using multiple sets of training data, and each of the multiple sets of training data has a matching score matrix. Each set includes: a sample elbow insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; according to the matching score matrix, a target candidate area in the elbow insertion image is determined, wherein the target candidate area includes each elbow; characteristic information of each elbow in the target candidate area is obtained, wherein the characteristic information includes: position information of each elbow, angle information of each elbow; and the elbow insertion detection results of the two air conditioners are obtained according to the matching score matrix and the characteristic information.
[0108] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: when it is determined that the air conditioner is transported to the elbow insertion area, the material clamping component of the elbow insertion area is controlled to clamp the elbow and insert the elbow into the air conditioner; after it is determined that the elbow is inserted into the air conditioner, the image acquisition device of the elbow insertion area is controlled to start to capture the elbow insertion image of the air conditioner.
[0109] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for executing the following steps: obtaining multiple sets of training data including sample elbow insertion images and sample matching score matrices; training the multiple sets of training data to obtain an elbow insertion detection model.
[0110] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: performing image preprocessing on the sample elbow insertion image and the template image to obtain the preprocessed sample elbow insertion image and the template image, wherein the template image is a top view image of the elbow; initializing the initial angle parameters, matching parameters and matching threshold of the elbow in the sample elbow insertion image, wherein the matching parameters are used to determine whether the sample elbow insertion image matches the template image, and the matching threshold is used to determine whether the template image is similar to the sample elbow image. The threshold score; randomly initializing to obtain a mask matrix of the same size as the sample elbow insertion image and the template image, with The sample elbow insertion image and the template image are filtered by using a mask matrix to obtain the filtered sample elbow insertion image and the template image; the multi-scale features of the filtered sample elbow insertion image are extracted; the feature vectors at different positions of the sample elbow insertion image are subjected to regional discrete weighted processing according to the multi-scale features to remove the feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow to obtain the retained feature vectors; sparse feature selection is performed on the retained feature vectors to obtain the sparse feature selection results; the above steps are repeated until the error value between the obtained predicted matching score matrix and the sample matching score matrix is less than the error threshold to obtain the elbow insertion detection model.
[0111] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for performing the following steps: setting a selection step and a scaling factor of a template image; adjusting the angle and size of the template image according to the selection step and the scaling factor to obtain an adjusted template image; and generating an elbow insertion template library according to the template image and the adjusted template image.
[0112] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code for performing the following steps: randomly initializing a first mask matrix and a second mask matrix consisting of 0 and 1; filtering the template image using the first mask matrix to narrow the search area of the sample elbow insertion image; filtering the sample elbow insertion image using the second mask matrix to filter out interference areas in the sample elbow insertion image, wherein the interference area includes at least the background area of the sample elbow insertion image.
[0113] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: continuously downsampling and upsampling the filtered sample elbow insertion images to obtain sample elbow insertion images of different resolutions; performing feature extraction on the sample elbow insertion images of different resolutions to obtain multi-scale features.
[0114] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: performing regional segmentation on multi-scale features to obtain position information of each region in the sample elbow insertion image; assigning corresponding weights to different positions based on the position information; performing weighted summation processing on different positions based on the weights to remove feature vectors in the sample elbow insertion image that are not related to the geometric constraints of the elbow, and obtain retained feature vectors.
[0115] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: when the elbow insertion detection result indicates that there is a missing elbow in the two air conditioners, determine the coordinate information and angle information of the missing elbow according to the elbow insertion detection result; control the material clamping component to clamp the elbow according to the coordinate information and the angle information, and insert the elbow to the missing elbow position corresponding to the coordinate information according to the angle information.
[0116] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0117] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0118] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0120] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0122] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for detecting elbow insertion of two air conditioners. It is characterized in that include: Acquire an elbow-insertion image of two air conditioners, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow-insertion image is an image of the two air conditioners with the elbow inserted; The elbow insertion image is input into the elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample elbow insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; Determine a target candidate region in the elbow insertion image according to the matching score matrix, wherein the target candidate region includes each of the elbows; Acquire feature information of each of the elbows in the target candidate area, wherein the feature information includes: position information of each of the elbows and angle information of each of the elbows; The elbow insertion detection results of the two air conditioners are obtained according to the matching score matrix and the characteristic information.
2. The elbow plug-in detection method for two air conditioners according to claim 1, It is characterized in that Get the elbow plug-in images of the two air conditioners, including: When it is determined that the two air conditioners are transported to the elbow insertion area, controlling the material clamping component of the elbow insertion area to clamp the elbow and insert the elbow into the two air conditioners; After determining that the elbow is inserted into the two air conditioners, the image acquisition device in the elbow insertion area is controlled to start to capture the elbow insertion image of the two air conditioners.
3. The elbow plug-in detection method for two air conditioners according to claim 1, It is characterized in that Before inputting the elbow insertion image into the elbow insertion detection model to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, the method further includes: Acquire the plurality of sets of training data including the sample elbow insertion images and the sample matching score matrix; The multiple groups of training data are trained to obtain the elbow insertion detection model.
4. The elbow plug-in detection method for two air conditioners according to claim 3, It is characterized in that Training the multiple sets of training data to obtain the elbow insertion detection model includes: Performing image preprocessing on the sample elbow insertion image and the template image to obtain the preprocessed sample elbow insertion image and the template image, wherein the template image is a top view image of the elbow; Initialize the initial angle parameters, matching parameters and matching threshold of the elbow in the sample elbow insertion image, wherein the matching parameters are used to determine whether the sample elbow insertion image matches the template image, and the matching threshold is used to determine the threshold score of whether the template image is similar to the sample elbow image; Randomly initialize to obtain a mask matrix of the same size as the sample elbow insertion image and the template image, and use the mask matrix to filter the sample elbow insertion image and the template image to obtain the filtered sample elbow insertion image and the template image; Extracting multi-scale features of the sample elbow insertion image after filtering; Performing region-by-region discrete weighted processing on feature vectors at different positions of the sample elbow insertion image according to the multi-scale features, so as to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, and obtain retained feature vectors; Performing sparse feature selection on the retained feature vector to obtain a sparse feature selection result; The above steps are repeated until the error value between the obtained prediction matching score matrix and the sample matching score matrix is less than the error threshold, thereby obtaining the elbow insertion detection model.
5. The elbow plug-in detection method for two air conditioners according to claim 4, It is characterized in that Also includes: Setting the selection step size and scaling factor of the template image; Adjust the angle and size of the template image according to the selected step size and the scaling factor to obtain the adjusted template image; An elbow insertion template library is generated according to the template image and the adjusted template image.
6. The elbow plug-in detection method for two air conditioners according to claim 4, It is characterized in that The mask matrix includes a first mask matrix and a second mask matrix, and is randomly initialized to obtain a mask matrix of the same size as the sample elbow insertion image and the template image, so as to filter the sample elbow insertion image and the template image using the mask matrix to obtain the filtered sample elbow insertion image and the template image, including: Randomly initialize the first mask matrix and the second mask matrix consisting of 0 and 1; Using the first mask matrix to filter the template image to narrow the search area of the sample elbow insertion image; The sample elbow insertion image is filtered using the second mask matrix to filter out interference areas in the sample elbow insertion image, wherein the interference areas at least include background areas of the sample elbow insertion image.
7. The elbow plug-in detection method for two air conditioners according to claim 4, It is characterized in that Extracting multi-scale features of the sample elbow insertion image after filtering, including: Performing continuous downsampling and upsampling on the sample elbow insertion image after filtering to obtain the sample elbow insertion images with different resolutions; Feature extraction is performed on the sample elbow insertion images with different resolutions to obtain the multi-scale features.
8. The elbow plug-in detection method for two air conditioners according to claim 4, It is characterized in that According to the multi-scale features, feature vectors at different positions of the sample elbow insertion image are subjected to regional discrete weighted processing to remove feature vectors in the sample elbow insertion image that are irrelevant to the geometric constraints of the elbow, and obtain retained feature vectors, including: Performing regional segmentation on the multi-scale features to obtain position information of each region in the sample elbow insertion image; assigning corresponding weights to the different positions according to the position information; A weighted summation process is performed on the different positions according to the weights to remove feature vectors irrelevant to the geometric constraints of the elbow in the sample elbow insertion image, so as to obtain a retained feature vector.
9. The elbow insertion detection method for two air conditioners according to any one of claims 1 to 8, It is characterized in that After obtaining the elbow insertion detection results of the two air conditioners according to the matching score matrix and the characteristic information, the method further includes: When the elbow insertion detection result indicates that there is a missing elbow insertion in the two air conditioners, determining the coordinate information and angle information of the missing elbow insertion according to the elbow insertion detection result; The material clamping component is controlled to clamp the elbow according to the coordinate information and the angle information, and the elbow is inserted into the elbow missing position corresponding to the coordinate information according to the angle information.
10. An elbow plug-in detection device for two air conditioners, It is characterized in that include: A first acquisition module is used to acquire an elbow-insertion image of two air conditioners, wherein the two air conditioners include: an evaporator and a condenser of the air conditioner, and the elbow-insertion image is an image of the two air conditioners with the elbow inserted; An input module, used for inputting the elbow insertion image into the elbow insertion detection model, so as to process the elbow insertion image using the elbow insertion detection model to obtain a matching score matrix of the elbow insertion image, wherein the elbow insertion detection model is a model obtained by machine learning training using multiple sets of training data, each of the multiple sets of training data includes: a sample elbow insertion image and a sample matching score matrix corresponding to the sample elbow insertion image, the matching score matrix is used to represent the insertion score of each elbow in the elbow insertion image, and the score value of the insertion score is used to represent whether each elbow is correctly plugged in; A first determination module is used to determine a target candidate region in the elbow insertion image according to the matching score matrix, wherein the target candidate region includes each of the elbows; A second acquisition module is used to acquire feature information of each of the elbows in the target candidate area, wherein the feature information includes: position information of each of the elbows and angle information of each of the elbows; The third acquisition module is used to obtain the elbow insertion detection results of the two air conditioners according to the matching score matrix and the characteristic information.
11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the elbow insertion detection method for two devices of the air conditioner according to any one of claims 1 to 9.
12. A processor, It is characterized in that The processor is used to run a program, wherein the program, when running, executes the elbow insertion detection method for two air conditioners according to any one of claims 1 to 9.
13. A computer program product comprising computer instructions, It is characterized in that When the computer instructions are executed by the processor, the elbow insertion detection method for two air conditioners described in any one of claims 1 to 9 is performed.
