Method for monitoring installation process of air conditioning unit, computing device, apparatus, monitoring system and computer readable storage medium

Abstract

The present application relates to air conditioner installation technical field, provide a kind of air conditioning unit installation process monitoring method and monitoring system.Therein, monitoring method includes the following steps: image acquisition step, obtains one or more images of air conditioning unit installation scene;Component identification step, from the image obtained in image acquisition step, identify one or more relevant components used in air conditioning unit installation process;Parameter identification step, based on one or more features of relevant component identified by component identification step, determine one or more parameters of relevant component;Parameter comparison step, compare the parameters determined in parameter identification step with the database of pre-stored relevant component, to monitor air conditioning unit installation process.The monitoring method of the present application utilizes image recognition technology, can effectively capture a series of installation positions, appearance structure and other aspects of features reflecting air conditioner installation quality, and then be distinguished with low cost and high efficiency by less calculation amount.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of air conditioner installation, and more particularly, to a monitoring method for air conditioner unit installation process. BACKGROUND

[0002] Air conditioner unit installation has a great influence on the safety and quality of the use of air conditioner unit in the future, and there is a saying in the air conditioning industry that "three parts of quality, seven parts of installation". However, the installation process and quality test after installation of the air conditioner are often completed by the installation personnel themselves, and such installation process management and test method is easy to cause many problems. On the one hand, the management of the installation process and the quality test completely depend on the experience of the installation personnel, and if the execution of the installation quality standard is to be supervised, it is necessary to send people to conduct spot checks, which not only consumes a large amount of labor cost, but also the coverage of the supervision work is narrow due to the limited number of spot check samples; on the other hand, the use of materials completely depends on the operation of the installation personnel, and there is a lack of supervision means.

[0003] The installation process of the air conditioner unit has many characteristics such as a large number of auxiliary materials needed, complicated installation steps, etc., and under many conditions, the technical personnel in the field have been difficult to obtain a method which can complete the installation process monitoring at low cost and high efficiency. SUMMARY

[0004] The inventor found through careful observation and research that in the installation process, the installation quality of the related components can be directly reflected in a series of computer vision level distinguishable features such as installation position, appearance structure, etc., and then the features can be distinguished at low cost and high efficiency.

[0005] The monitoring method for air conditioner unit installation process provided by the present application comprises the following steps: an image acquisition step of acquiring one or more images of an air conditioner unit installation scene; a component identification step of identifying one or more related components used in the air conditioner unit installation process from the images acquired in the image acquisition step; a parameter identification step of determining one or more parameters of the related components based on one or more features of the related components identified in the component identification step; and a parameter comparison step of comparing the parameters determined in the parameter identification step with a pre-stored database of related components to monitor the air conditioner unit installation process.

[0006] By using image recognition technology, the components used in the air conditioner installation process can be identified and recorded, and after comparison with the pre-stored database, it is judged whether the components used in the air conditioner installation process meet the preset standard and a reminder or record is given, so as to monitor the installation process of the air conditioner unit. Therefore, the installation process of the air conditioner unit can be monitored at low cost and high efficiency with less calculation amount.

[0007] In an optional technical solution of the present application, the parameter is the type and / or performance index of the relevant component. In the parameter comparison step, whether the specified relevant component is actually used in the air conditioning unit installation scene is determined by comparing the parameter with the database. On the one hand, it ensures that the unified installation quality standard is implemented, and on the other hand, it can record and supervise the material use behavior of the installation personnel.

[0008] In an optional technical solution of the present application, the relevant component is a pressure container. Preferably, it is a refrigerant tank. Since the type of refrigerant and the filling operation management are extremely critical links in the air conditioning installation process, they need to be managed in particular. By obtaining the image of the refrigerant tank, the correct filling and use of the refrigerant can be managed and monitored intuitively and accurately to ensure the quality of air conditioning installation.

[0009] At the same time, the refrigerant tank is a pressure container, the bottle body structure has a curvature, and the bottle body profiles of different types of refrigerant tanks are significantly different. By pre-setting corresponding image recognition conditions, the computational load of image recognition can be effectively reduced, and classification recognition can be performed at low cost and high efficiency.

[0010] In an optional technical solution of the present application, the image acquisition step further comprises: acquiring a first image of a first face of the pressure container; and / or acquiring a second image of a second face of the pressure container.

[0011] In an optional technical solution of the present application, the first face is a side face of the pressure container, and the features include at least one of the curvature of the top end profile line of the pressure container, the diameter of the pressure container, the position or pipe diameter of the inlet / outlet pipe of the pressure container, color, text, and height. In an optional technical solution of the present application, the second face is a top face of the pressure container, and the features include at least one of the diameter of the pressure container, the position or pipe diameter of the inlet / outlet pipe of the pressure container, color, and text. The images of refrigerant tanks of different angles are suitable for being identified using different features, and reasonable combination and selection of the above-mentioned features can effectively improve the speed and accuracy of identification.

[0012] In an optional technical solution of the present application, the component identification step and / or the parameter identification step further comprise: providing an image data set, the images in the image data set being labeled according to whether they contain the relevant component; training a classification model based on the image data set to obtain a trained classification model; and classifying the images obtained in the image acquisition step based on the trained classification model. Training the classification model using the image data set can effectively improve the overall classification accuracy of the classification model.

[0013] In an optional technical solution of the present application, the trained classification model classifies the image obtained in the image obtaining step according to the matching degree of the image obtained in the image obtaining step and the images in the image dataset. The classification based on the matching degree can more comprehensively identify the object, and reduce the operation amount required in the feature extraction process, and the running speed is faster.

[0014] In an optional technical solution of the present application, the monitoring method further comprises: uploading and saving the image obtained in the image obtaining step in the image dataset, and labeling the image obtained in the image obtaining step according to the evaluation result to obtain an updated image dataset; and further training the trained classification model based on the updated image dataset. The monitoring method can continuously train the classification model using the constantly updated image dataset, and effectively improve the accuracy of image recognition.

[0015] In an optional technical solution of the present application, the relevant component is a pressure gauge. The vacuum extraction and vacuum degree maintenance and confirmation before refrigerant filling are extremely critical links in the air conditioner installation process, and need to be managed and monitored. By obtaining the image of the pressure gauge, the vacuum extraction and vacuum degree maintenance before refrigerant filling can be managed and monitored intuitively and accurately to ensure the quality of air conditioner installation.

[0016] In an optional technical solution of the present application, the image obtaining step further comprises: obtaining a third image related to the installation scene of the first air conditioning unit for the pressure gauge at a first time; obtaining a fourth image related to the installation scene of the second air conditioning unit for the pressure gauge at a second time; and the parameter identifying step further comprises: determining a first pressure value of the pressure gauge based on one or more features of the pressure gauge in the third image; determining a second pressure value of the pressure gauge based on one or more features of the pressure gauge in the fourth image; and determining the change of the pressure value of the pressure gauge based on the first pressure value, the second pressure value, the time point value of the first time, and the time point value of the second time. In this way, the monitoring method can analyze the dynamic change process of the pressure gauge reading by using multiple images taken at different time points, i.e., at different air conditioning unit installation scenes, and further analyze the vacuum degree and sealing condition of the pipeline based on the dynamic change process of the pressure gauge reading.

[0017] In an optional technical solution of the present application, the first time is the starting time point of the vacuum extraction of the installed air conditioning unit. In an optional technical solution of the present application, the second time is the ending time point of the vacuum extraction of the installed air conditioning unit or the standing time point after a specified time after the vacuum extraction. By identifying the change rate of the pressure value in the vacuum extraction process or the standing process after the vacuum extraction process, the pipeline sealing condition of the air conditioning unit can be quantitatively indicated.

[0018] In an optional technical solution of the present application, the features include at least the range information of the pressure gauge and the deflection angle information of the pointer of the pressure gauge.

[0019] In an optional technical solution of the present application, the related component is an electric wire, and the parameter includes the diameter of the electric wire.

[0020] In the installation process of an air conditioner, it is necessary to confirm and manage whether the electric wire used conforms to the standard, and the diameter of the electric wire is an important management index. However, the image obtained by the installation personnel by taking a photo of the electric wire does not have size information, so it is difficult to directly determine the diameter of the electric wire by using image recognition. In an optional technical solution of the present application, the related component further includes a circuit breaker unit, and in the image acquisition step, the image information of the circuit breaker unit and the electric wire is contained in one image; the monitoring method of the air conditioner unit installation process further includes the following steps: based on the image, determining the number of first pixels occupied by the circuit breaker unit in the width direction and the number of second pixels occupied by the electric wire in the diameter direction; based on the number of first pixels and the number of second pixels, referring to the data of the circuit breaker unit in the width direction as a standard accessory, to determine the diameter of the electric wire. Through the above method, a ruler is established by using the width of the circuit breaker unit connected to the electric wire, and the diameter of the electric wire is determined by means of the ruler.

[0021] In an optional technical solution of the present application, the electric wire includes a first color electric wire and a second color electric wire, and the image obtained in the image acquisition step includes a color image; the monitoring method of the air conditioner unit installation process further includes the following steps: processing the color image to obtain a first color channel image and a second color channel image; determining the diameter of the first color electric wire according to the first color channel image; determining the diameter of the second color electric wire according to the second color channel image. Generally, the types of colors used by electric wires are relatively uniform, and through the above image processing method based on colors, different color electric wires can be easily identified from the image and their diameters can be determined.

[0022] In an optional technical solution of the present application, the features are the text information printed on the surface of the related component.

[0023] In an optional technical solution of the present application, the monitoring method of the air conditioner unit installation process further includes the following steps: the comparison result sending step feeds back the comparison result obtained in the parameter comparison step to the installation personnel or a third party preset in advance. The above method enables the installation personnel to know whether the installation process or the used component conforms to the regulations, or helps the user or the background management personnel to know and monitor the installation progress and the qualified and compliant situation.

[0024] The application further provides a computing device for monitoring an air conditioning unit installation process and a monitoring system comprising the same, the computing device comprising: at least one processor; and a memory communicatively coupled with the at least one processor and readable by the at least one processor, and having stored therein processor-readable instructions that, when executed by the at least one processor, cause the at least one processor to perform the following operations: an image acquisition step of acquiring one or more images of an air conditioning unit installation scene; a component identification step of identifying, from the images acquired in the image acquisition step, one or more relevant components used in the air conditioning unit installation process; a parameter identification step of determining one or more parameters of the relevant components based on one or more features of the relevant components identified in the component identification step; and a parameter comparison step of comparing the parameters determined in the parameter identification step with a pre-stored database of the relevant components to monitor the air conditioning unit installation process.

[0025] In an optional technical solution of the application, the parameters are types and / or performance indicators of the relevant components.

[0026] In an optional technical solution of the application, the monitoring system of the air conditioning unit installation process further comprises a mobile terminal, and the mobile terminal comprises: an image acquisition device configured to acquire one or more images of an air conditioning unit installation scene; and a wireless communication module capable of uploading the images acquired by the image acquisition device through wireless transmission.

[0027] The application further provides a device for monitoring an air conditioning unit installation process, comprising: an image acquisition unit configured to acquire one or more images of an air conditioning unit installation scene; a component identification unit configured to identify, from the images acquired in the image acquisition step, one or more relevant components used in the air conditioning unit installation process; a parameter identification unit configured to determine one or more parameters of the relevant components based on one or more features of the relevant components identified in the component identification step; and a parameter comparison unit configured to compare the parameters determined in the parameter identification step with a pre-stored database of the relevant components to monitor the air conditioning unit installation process.

[0028] The application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps of the aforementioned monitoring method of the air conditioning unit installation process. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 FIG. 1 is a schematic diagram of a monitoring system of an air conditioning unit installation process in an embodiment of the application;

[0030] Figure 2 FIG. 2 is a flowchart of a monitoring method of an air conditioning unit installation process in an embodiment of the application;

[0031] Figure 3is a schematic diagram of a monitoring system for an air conditioning unit installation process in an embodiment of the present application;

[0032] Figure 4 is a flowchart of a machine learning-based image recognition process in an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of how to determine the diameter of an electric wire based on the number of pixels in an image in an embodiment of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0035] 100 - installer, 100T - mobile terminal, 102 - wireless communication module, 200 - server end, 202 - processor, 204 - memory, 206 - communication device, 208 - image recognition module, 210 - database, 212 - image data set, 3 - electric wire, 3a - first color electric wire, 3b - second color electric wire, 4 - circuit breaker unit. DETAILED DESCRIPTION

[0036] Hereinafter, a preferred embodiment of the present application will be described in detail with reference to the accompanying drawings. Figure 1 In addition, the embodiments of the present application are not limited to the following embodiments, and various embodiments within the technical concept of the present application can be adopted.

[0037] Embodiment 1

[0038] The present embodiment provides a monitoring method for an air conditioning unit installation process, and a monitoring system for executing the monitoring method.

[0039] Referring to Figure 1 , the monitoring system provided by the present embodiment includes mobile terminals 100T carried by installers 100 respectively and a server end 200.

[0040] In the present embodiment, the mobile terminal 100T is a smart phone with a camera (image acquisition device, not shown in the figure) and a wireless communication module 102, and is installed with a quality management application. The quality management application can obtain the use permission of the camera, the wireless communication module 102 and the read permission of the album data, and then can take pictures or videos using the camera, and read the taken images (which can be an image file such as a picture, multiple pictures or a video) from the album data and upload them to the server end 200 via the wireless communication module 102.

[0041] Accordingly, the installer 100 carrying the mobile terminal 100T can take pictures of the air conditioner installation scene when providing on-site service, and upload the pictures to the server 200 through the wireless communication module 102. It should be noted that in the present embodiment, the mobile terminal 100T provides the wireless communication module 102 as a cellular network module, which can be, for example, a long term evolution (LTE), code division multiple access (CDMA), wideband CDMA (WCDMA), or global system for mobile communications (GSM) network access, but in other embodiments of the present application, any other suitable wired or wireless (such as Bluetooth, wifi, etc.) transmission method can be used to provide network access.

[0042] The server 200 receives the images provided by the mobile terminal 100T through its communication device 206, calls the image recognition module 208 from the memory 204, and performs image recognition processing.

[0043] The image recognition module 208 can be implemented based on one or more of text recognition, digital image processing and recognition, or object recognition. In the present embodiment, the image recognition module 208 is implemented using text recognition, which can extract text information from the captured images by the image recognition module 208, and determine whether one or more images contain relevant components used in the air conditioner installation process based on the text information.

[0044] Taking the refrigerant tank as an example of the relevant component to be identified in the present embodiment, the monitoring method of the image recognition module 208 provided in the present embodiment is introduced. It should be noted that although the following embodiments are introduced by taking the identification of the refrigerant tank as an example, the monitoring method can also be applied to the installation quality monitoring of other relevant components used in the air conditioner installation process.

[0045] Reference Figure 2 The text recognition monitoring process specifically includes:

[0046] First, the installer 100 takes pictures of the refrigerant tank at a certain angle with specific information under the prompt of the quality management application of the mobile terminal 100T, and uploads the pictures to the server.

[0047] The processor 202 of the server is communicatively coupled to the memory 204, and by reading and running the processor-readable instructions in the memory, the following steps can be performed:

[0048] S01 image acquisition step, acquiring one or more images of the air conditioner installation scene.

[0049] Specifically, in the present embodiment, the server 200 receives the images of the air conditioner installation scene from the mobile terminal 100T of the installer 100 through its communication device 206.

[0050] The S02 component identification step identifies one or more components used in the air conditioning unit installation process from the image obtained in the S01 image acquisition step.

[0051] Specifically, the server 200 uses the image recognition module 208 in its computing device to perform optical character recognition on the uploaded image and extract text information from the image. In this embodiment, the extracted text information includes at least the text information on the surface of the refrigerant tank, such as “REFRIGERANT” and “22”. Based on the extracted text information, the relevant component used in the air conditioning unit installation process in the image can be determined. For example, in this embodiment, the refrigerant tank in the image can be determined based on the extraction of “REFRIGERANT”.

[0052] The S03 parameter identification step determines one or more parameters of the relevant component based on one or more features of the relevant component identified in the S02 component identification step.

[0053] By extracting other text information in the image, some parameters of the relevant component can be determined. In this embodiment, by reading the text “22” adjacent to “REFRIGERANT” in the image of the refrigerant tank, the image recognition module 208 can determine the type of the refrigerant tank.

[0054] In other embodiments of the present application, optical character recognition can be combined with other image recognition methods. For example, in some embodiments, the image recognition module 208 can further include a digital image processing and recognition module that can determine the relative positional relationship between characters, as well as the direction, size, font, etc. of the characters, based on the coordinates of the pixels of the characters in the image; then, based on the content of the characters, combined with the relative positional relationship between the characters, as well as the direction, size, font, etc. of the characters, the relevant component used in the air conditioning unit installation process is recognized from the image through comprehensive judgment. The combination of optical character recognition and other image recognition methods can significantly improve the accuracy of the image recognition module 208 in identifying the relevant component.

[0055] Through the above optical character recognition method, the image recognition module 208 can identify the refrigerant tank (the relevant component in this embodiment) from the image and determine that the type of the “refrigerant tank” is R22 (the parameter in this embodiment) based on the text information (the feature in this embodiment) printed on the surface of the refrigerant tank.

[0056] The S04 parameter comparison step compares the parameters determined in the S03 parameter identification step with the pre-stored database 210 of relevant components to monitor the air conditioning unit installation process.

[0057] The model information of the work order performed by the installer 100 in this on-site service is stored in the memory 204 of the monitoring system, and based on the model information, the monitoring system can search the pre-stored database 210 of related components to obtain the parameters of the related components that should be used for the model. In this embodiment, the monitoring system can search the pre-stored database 210 of related components based on the model information of the air conditioning unit to obtain the type of refrigerant tank that should be used for the model air conditioner. In some embodiments, the monitoring system can track the work order being performed by the installer 100, and determine the work order performed by the installer 100 in this on-site service by the positioning device of the mobile terminal 100T of the installer 100.

[0058] Then, the type of refrigerant tank that should be used is compared with the type of refrigerant tank actually used to determine whether the installer 100 used the specified refrigerant tank to monitor the installation process of the air conditioning unit. For example, in this embodiment, if the model information is searched to find that the R22 type refrigerant tank should be used for the model, it is determined to be qualified; if the model information is searched to find that the R290 type refrigerant tank should be used for the model, it is determined to be unqualified.

[0059] S05, the comparison result obtained in the S04 parameter comparison step is fed back to the installer or a third party set in advance.

[0060] In this embodiment, the result of comparing the type of refrigerant tank that should be used based on the model information with the type of refrigerant tank actually used based on the image recognition module 208 is sent to the installer 100 or a third party including the user. The above method not only enables the installer 100 to be aware of the situation that the installation quality does not meet the standard, but also helps the user to know and monitor the installation progress and qualified and compliant situation.

[0061] In some embodiments, if the related component is not recognized in one or more images uploaded on the mobile terminal 100T, the installer 100 can be notified of the result of the unsuccessful recognition, and the installer 100's mobile terminal 100T is sent a request for re-uploading. In some embodiments, the image recognition module 208 can process the images uploaded by the installer 100 in real time, and inform the installer 100 of the comparison result of the S04 parameter comparison step. The installer 100 can quickly adjust the installation process according to the comparison result, effectively improving the installation efficiency and installation quality.

[0062] By the above means, the monitoring system and the monitoring method provided by the embodiment can recognize and record the components used in the air conditioner installation process by using image recognition technology, and compare with the pre-stored database 210 to determine whether the components used in the air conditioner installation process meet the preset standards and give reminders or records, so as to monitor the installation process of the air conditioner unit. Therefore, the installation process of the air conditioner unit can be monitored with less calculation and at low cost and high efficiency.

[0063] Embodiment 2

[0064] The embodiment provides a monitoring system and a monitoring method which can be effectively applied to the refrigerant tank identification scene. The monitoring system and the monitoring method are designed and implemented based on the monitoring system and the monitoring method in embodiment one, and their processes and structures are similar but different. The differences will be described below. If no differences are explicitly stated from embodiment one, the processes or structures can be considered the same as those in embodiment one and will not be described again.

[0065] In the embodiment, since the refrigerant tank is a pressure vessel, the bottle body structure has a curvature and the bottle body profiles of different types of refrigerant tanks are obviously different. Therefore, taking the bottle body profile as a feature can effectively reduce the calculation amount of image recognition and classify images at low cost and high efficiency. In addition, the type of refrigerant and filling operation management are extremely critical links in the air conditioner installation process and need to be managed.

[0066] Reference Figure 3 Unlike embodiment one, the image recognition module 208 in the embodiment is designed and implemented based on object recognition. The monitoring method for the air conditioner unit installation process provided in the embodiment will be described below with the refrigerant tank as an example.

[0067] In the embodiment, multiple images of the refrigerant tank can be taken. The multiple images can be multiple images taken from different angles of the refrigerant tank, or can be a video obtained by taking the refrigerant tank from a moving camera. Specifically, the multiple images include at least a first image of a first face of the refrigerant tank and a second image of a second face of the refrigerant tank, wherein the first face is a side face of the refrigerant tank and the second face is a top face of the refrigerant tank.

[0068] In the embodiment, after the image of the refrigerant tank is taken, the quality management application of the mobile terminal 100T can notify the user to frame a certain area in the obtained image to mark the position of the relevant component to be identified, so as to help the image recognition module 208 improve the speed and accuracy of identification. When the image is uploaded to the server side, the framed area information will also be associated and uploaded to the server side 200.

[0069] In this embodiment, multiple images taken from multiple angles such as top view and side view are used for image recognition, and images of the refrigerant tank from different angles are suitable for recognition using different features. Among them, the first image taken from the side view, which indicates the side view of the refrigerant tank, is suitable for image recognition using features including but not limited to the following: curvature of the top end profile of the refrigerant tank, diameter of the refrigerant tank, position or pipe diameter of the inlet / outlet pipe of the refrigerant tank, color, text, height. The second image taken from above the refrigerant tank, which indicates the top view of the refrigerant tank, is suitable for image recognition using features including but not limited to the following: diameter of the refrigerant tank, position or pipe diameter of the inlet / outlet pipe of the refrigerant tank, color, text. Reasonable combination of the selected features can effectively improve the speed and accuracy of recognition.

[0070] In addition, in some embodiments, the recognition of related components can be performed by establishing a three-dimensional model of different types of refrigerant tanks, comparing multiple images from different angles with the three-dimensional model, or comparing images with three-dimensional structure information obtained by a 3D sensor or a depth camera with the three-dimensional model.

[0071] Reference Figure 4 In some other embodiments of the present application, a machine learning (deep) based object detection method can also be used for image recognition. Accordingly, the S02 component recognition step and the S03 parameter recognition step can be implemented in the same image recognition process, which specifically includes:

[0072] First, an image dataset 212 is provided, and the images in the image dataset 212 have been labeled according to whether they contain related components; then, based on the image dataset 212, a classification model is trained to obtain a trained classification model; finally, based on the trained classification model, the images obtained in the S01 image acquisition step are classified.

[0073] Providing the image dataset 212 requires collecting a certain number of images, some of which are images taken from different angles of refrigerant tanks, and the types of refrigerant tanks are also different; some of the images do not contain refrigerant tanks, and some of the images without refrigerant tanks also contain similar objects. Then, these images are manually labeled, for example, images without refrigerant tanks are labeled as "none"; images of R22 type refrigerant tanks are labeled as "R22"; images of R290 type refrigerant tanks are labeled as "R290"; and so on. Then, these images are packaged and combined to form the image dataset 212.

[0074] In some embodiments, the image recognition module 208 can extract the outline shape of the coolant tank from the one or more images obtained in the S01 image acquisition step, and compare the outline shape with the images in the image dataset 212 to obtain a matching degree of the outline shape. Based on the matching degree, the one or more images obtained in the S01 image acquisition step are classified, and the images are labeled according to the results of the image classification. The type of the label can at least indicate whether the coolant tank is present in the images. In some embodiments, the type of the label can also indicate the brand, type, and other parameters of the coolant tank in the images.

[0075] In addition, in some embodiments, the content extracted from the images can also include texture, color, and other information of the coolant tank, and the one or more images obtained in the S01 image acquisition step are classified based at least in part on the matching degree of the texture, color, and other information.

[0076] With reference to the foregoing Figure 4 To improve the recognition speed and accuracy of the image recognition module, the labeled image dataset 212 can be used to train and test the classification model of the image recognition module 208. Further, in some embodiments, the image dataset 212 can also be continuously updated using the images obtained in the S01 image acquisition step, and the continuously updated image dataset 212 can be used to continuously train the classification model.

[0077] Specifically, the above-mentioned continuous updating monitoring method includes uploading and saving the images obtained in the S01 image acquisition step in the image dataset 212, and labeling the images obtained in the S01 image acquisition step as positive or negative samples according to the evaluation results to obtain an updated image dataset, and further training the trained classification model based on the updated image dataset.

[0078] In some embodiments, the evaluation result can be an evaluation result such as similarity made by the trained classification model in the classification process of the images. When the similarity exceeds a threshold value, it can be considered that the relevant components are recognized from the images, and is determined as a “positive result”, and the images are labeled as positive samples. When the similarity is lower than the threshold value, it can be considered that the relevant components are not recognized from the images, and is determined as a “negative result”, and the images are labeled as negative samples. The positive and negative sample labels of the images can be used in the training process of the classification model to improve the classification accuracy of the classification model.

[0079] In this way, the monitoring method can continuously train the classification model using the continuously updated image dataset, and effectively improve the accuracy of image recognition.

[0080] Embodiment 3

[0081] The monitoring system and monitoring method provided by the present embodiment can be effectively applied to the wire identification scene, and are designed and implemented based on the monitoring system and monitoring method in Embodiment One, and have similar but different processes and structures. The differences will be described below, and if no differences are explicitly stated, the processes and structures can be considered the same as those in Embodiment One and will not be described again.

[0082] In the air conditioner installation process, it is necessary to confirm and manage whether the used wire 3 meets the standard, and the diameter of the wire is an important management index. However, the image obtained by the installer by taking a photo of the wire 3 does not have size information, so it is difficult to directly determine the diameter of the wire 3 by image recognition.

[0083] To solve the above problems, the present embodiment provides a monitoring system and monitoring method suitable for a wire identification scene, which uses the width of the circuit breaker unit 4 connected to the wire 3 to establish a ruler to measure and determine the diameter of the wire 3 (the "parameter" in the present embodiment).

[0084] Specifically, the monitoring method provided by the present embodiment includes at least one image in the one or more images obtained in the S01 image acquisition step, which simultaneously contains the circuit breaker unit 4 and the wire 3. The installer needs to take a photo of the circuit breaker unit 4 and the wire 3 connected to the circuit breaker unit 4 in the same image, and upload the image to the server end 200. The quality management application can prompt the installer 100 to ensure that the object distance of the camera and the circuit breaker unit 4 and the wire 3 is basically the same during the shooting process. In this way, the image of the circuit breaker unit 4 and the wire 3 in or basically in the same object plane can be obtained. Since the width of the circuit breaker unit 4 has a standard, the width of the circuit breaker unit 4 with the same modulus is also the same, and the width of the circuit breaker unit 4 can be determined according to the modulus identification of the circuit breaker unit 4. Generally speaking, the width of the circuit breaker unit 4 with 1P, i.e., 1 modulus, is 18mm, and the width of the circuit breaker unit 4 with 2P-4P modulus is 2-4 times of 18mm, respectively. In some embodiments, the modulus of the circuit breaker unit 4 in the image can be determined by image recognition, and the width of the circuit breaker unit 4 can be determined based on the modulus, or the modulus information of the circuit breaker unit 4 can be obtained when the installer 100 takes the image and uploaded.

[0085] Reference Figure 5 In the present embodiment, the camera of the mobile terminal 100T of the installer 100 is a color camera, and the obtained image is a color image. The recognition of the color image is performed by the processor 202 of the server end 200 by loading the image recognition module 208 from the storage 204, and specifically includes the following steps:

[0086] Based on the brightness of the first color (red), the color image obtained in the S01 image acquisition step is binarized to obtain the first color channel image; based on the brightness of the second color (blue), the color image obtained in the S01 image acquisition step is binarized to obtain the second color channel image; features of at least a portion of the contour shape of the circuit breaker unit 4 and the wire 3 are extracted and closed-loop operation is performed.

[0087] Specifically, in this embodiment, the color image is binarized based on the RGB values ​​of each pixel in the color image, namely the brightness values ​​of red, green, and blue. Based on the brightness of red, pixels with R values ​​higher than a threshold are assigned a value of 1, while pixels with R values ​​lower than a threshold are assigned a value of 0, forming a red channel image. The red channel image can easily identify the red wire—the live wire (first color wire 3a) in the image. Based on the brightness of blue, pixels with B values ​​higher than a threshold are assigned a value of 1, while pixels with B values ​​lower than a threshold are assigned a value of 0, forming a blue channel image. The blue channel image can easily identify the blue wire—the neutral wire (second color wire 3b) in the image.

[0088] Based on the aspect ratio of circuit breaker unit 4, the outline of circuit breaker unit 4 is located, and the number N of the first pixels occupied by circuit breaker unit 4 in its width direction is obtained. pixel1 .

[0089] Circuit breaker unit 4 is typically white, and during installation, a gap is left between adjacent circuit breaker units 4, with the outline of the gap appearing as a black shadow. Therefore, the outline of circuit breaker unit 4 can be easily identified using parameters such as grayscale and brightness. Furthermore, based on the aspect ratio of circuit breaker unit 4, outline positioning can be achieved more accurately.

[0090] Identify the first-color wire 3a from the first-color channel image and determine the number N of second pixels N occupied by the first-color wire 3a in its diameter direction. pixel2(3a) Identify the second-color wire 3b from the second-color channel image and determine the number N of second pixels N occupied by the second-color wire 3b in its diameter direction. pixel2(3b) .

[0091] Based on the number N of the first pixel pixel1 The number N of the second pixel pixel2 (Features), determine the diameter D(D) of the first colored wire 3a and the second colored wire 3b. 3a D 3b (parameters), where the number of second pixels includes the number N of second pixels corresponding to the first color wire 3a. pixel2(3a) The number N of the second pixels corresponding to the second-color wire 3b pixel2(3b) .

[0092] In some embodiments, for a single-mode circuit breaker unit, the diameter of the electric wire can be calculated by the following formula:

[0093]

[0094] where D is the diameter of the electric wire, N is the mode number of the circuit breaker unit, N pixel1 is the number of first pixels, and N pixel2 is the number of second pixels.

[0095] In the present embodiment, with reference to Figure 5 for a multi-mode (2-mode in the present embodiment) circuit breaker unit, the following formula can be used for calculation:

[0096]

[0097] where D is the diameter of the electric wire, N is the mode number of the circuit breaker unit, N pixel1 is the number of first pixels, and N pixel2 is the number of second pixels.

[0098] After the determination of the diameter D of the electric wire is completed, the diameter D of the electric wire is compared with the diameter parameter in the specification parameter of the electric wire 3 that should be used, which is queried from the database 210, and if it is within an acceptable reasonable range, it is determined that the diameter of the electric wire used is qualified.

[0099] In the above manner, the present embodiment uses the circuit breaker unit 4 appearing in the same image as the electric wire 3 as a size scale of the electric wire 3, and further can accurately estimate the diameter of the electric wire 3 using image recognition.

[0100] The parameters of the monitoring system and method of the air conditioning unit installation process provided by the first embodiment to the third embodiment are all types and / or performance indicators of related components, and in the S04 parameter comparison step, whether the specified related component is actually used in the air conditioning unit installation scene is determined by comparing the parameters with the database 210. In the above manner, on the one hand, it ensures that the unified installation quality standard can be implemented, and on the other hand, it can supervise the use behavior of the installation personnel 100.

[0101] Embodiment 4

[0102] The present embodiment provides a monitoring system and a monitoring method that can be effectively applied to a pressure gauge identification scene, which are both designed and implemented based on the monitoring system and the monitoring method in the first embodiment, and their processes and structures are similar but different. The differences are described below, and if no differences are explicitly stated from the first embodiment, the processes or structures can be considered the same as those in the first embodiment and will not be described again.

[0103] The vacuum extraction before refrigerant filling and the maintenance and confirmation of the vacuum degree are extremely critical links in the air conditioner installation process, and need to be managed and monitored. By obtaining the image of the pressure gauge, the vacuum extraction before refrigerant filling and the maintenance of the vacuum degree can be managed and monitored directly and accurately to ensure the quality of air conditioner installation. Before that, the installer usually uses soap water to observe the bubble situation on the surface of the pipeline to judge whether the pipeline of the air conditioning unit has a leakage point. It is not only time-consuming and laborious, but also cannot quantitatively judge the sealing condition of the pipeline. The embodiment provides an image processing method based on a visual library such as openCV, which can efficiently and quantitatively indicate the sealing condition of the pipeline.

[0104] The image processing method based on openCV can directly read the pressure gauge reading from the image. Specifically, first, the approximate outline of the pressure gauge in the image is extracted by the image processing method of openCV; then, angle correction is performed, and the position of the pointer is identified based on the corrected pressure gauge; and then, the reading (parameter) of the pressure gauge is calculated according to the range information and the pointer deflection angle information of the pressure gauge.

[0105] Specifically, in the embodiment, the S01 image recognition step further includes: at a first time, acquiring a third image related to the first air conditioning unit installation scene for the pressure gauge; and at a second time, acquiring a fourth image related to the second air conditioning unit installation scene for the pressure gauge. The S03 parameter recognition step further includes: determining a first pressure value of the pressure gauge based on the range information and the pointer deflection angle information of the pressure gauge in the third image; and determining a second pressure value of the pressure gauge based on the range information and the pointer deflection angle information of the pressure gauge in the fourth image. Based on the first pressure value, the second pressure value, the time point value of the first time, and the time point value of the second time, the change of the pressure value of the pressure gauge is determined.

[0106] In some embodiments, the first time is the starting time point of the vacuum extraction of the installed air conditioning unit, and the second time is the ending time point of the vacuum extraction of the installed air conditioning unit. By identifying the change rate of the pressure value during the vacuum extraction, the sealing condition of the pipeline of the air conditioning unit can be quantitatively indicated.

[0107] In some other embodiments, the first time is an end time point of ending vacuumizing of the installed air conditioning unit, and the second time is a standing time point after a specified time after the ending of the vacuumizing. The installer can take the first photo and upload at the end of the vacuumizing, and return to the installation site after the specified time interval, take the second photo of the pressure gauge and upload again. Through the difference between the second pressure value and the first pressure value, the pressure value change of the air conditioning unit in this period of time can be reflected; through the difference between the time point value of the second time and the time point value of the first time, the length of the elapsed time can be reflected; through the pressure value change and the length of the elapsed time, the pressure value change rate of the air conditioning unit can be indicated, and then the pipeline sealing condition of the air conditioning unit can be quantitatively indicated.

[0108] In the above manner, the present embodiment can efficiently complete the quantitative judgment of the pipeline sealing condition of the air conditioning unit through the image recognition manner with the help of the reading recognition of the pressure gauge.

[0109] Terminology and Explanation

[0110] In the present embodiment, the mobile terminal 100T used by the installer 100 can be any suitable mobile terminal with communication function, such as a mobile phone, a MP3 player, a tablet computer, a notebook computer, a PDA, etc.

[0111] In the present embodiment, the term “feature” refers to the image information (or data information derived from the image information) that can be distinguished by the image recognition technology and can help analyze the target (related component), and the “parameter” of the related component can be the textual or numerical information reflecting the characteristics of the related component, or the textual or numerical information reflected by the related component.

[0112] It should be noted that, unless explicitly stated, the “acquiring image” referred to herein should be understood as at least including the following two cases: taking an image by using an image acquisition device such as a camera; reading an image file by using an input / output interface or a network interface.

[0113] In the embodiments of the present application, the "installation process" refers to the construction process of installing the air conditioning unit, and at least includes a preparation stage of relevant components, a construction stage of work, and an installation quality evaluation stage. In some embodiments, the monitoring method can be performed in the preparation stage of relevant components, and can be performed by taking pictures of the displayed components to be installed (such as pressure vessels and wires) to help determine whether the specified components are actually used by the installation personnel; in other embodiments, the monitoring method can also be performed in the construction stage of work, for example, when the air conditioning unit is being evacuated, a pressure gauge for detecting the vacuum degree of the air conditioning unit is taken to achieve feedback on the installation quality by reading the readings; in other embodiments, the monitoring method can also be performed in the installation quality evaluation stage, for example, taking pictures of the installed wires, pipelines, etc. to evaluate the installation quality or performance parameters of the wires and pipelines.

[0114] The computing device of the server end 200 can be a conventional computer system, an embedded controller, a laptop computer, a server, a mobile device, a smart phone, a set-top box, a kiosk, a car information system, one or more processors associated with a system, a custom machine, any other hardware platform, or a combination of the above computing devices. The computing device can also be configured as a distributed system operating via multiple computing devices interconnected via a data network or a bus system. In some embodiments of the present application, the server end 200 can also be integrated into the same mobile terminal as the mobile terminal 100T used by the installation personnel 100, and the image recognition process can be completed offline using the image recognition module of the mobile terminal.

[0115] The memory 204 can include non-volatile memory, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), flash memory (FLASH), or any other device capable of storing program instructions or data with or without the application of power. The memory can also include volatile memory, such as random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), and synchronous dynamic random access memory (SDRAM). Other types of RAM can also be used to implement the memory. The memory can be implemented using a single memory module or multiple memory modules. Although the memory is depicted as part of the computing device, those skilled in the art can recognize that the memory can be separate from the computing device without departing from the scope of the subject technology.

[0116] The processor 202 can be a general processor, a processor core, a multi-processor, a reconfigurable processor, a microcontroller, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a programmable logic device (PLD), a controller, a state machine, gated logic, discrete hardware components, or any other processing unit, combination of one or more of the above, or the like.

[0117] Furthermore, references herein to "embodiment" means that a particular feature, structure, function, or characteristic being discussed can be included in at least one embodiment of the application. Thus the appearances of the phrase "in one embodiment" or "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, functions, or characteristics can be combined in any suitable manner in one or more embodiments. For example, the text recognition method in one embodiment can be combined with the object recognition, digital image processing recognition, etc. in other embodiments, as long as the embodiments are not mutually exclusive.

[0118] Also, the term "module" can include one or more hardware or software elements configured to facilitate the performance of the various methods and processing functions presented herein, and can also include one or more sequences of instructions stored as software or firmware in association with a memory.

[0119] Up to now, the technical solutions of the present application have been described in conjunction with the accompanying drawings, but those skilled in the art can easily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical solutions after the changes or replacements will all fall within the protection scope of the present application.

Claims

1. A method for monitoring the installation process of an air conditioning unit, characterized in that, Includes the following steps: The image acquisition step involves acquiring one or more images of the air conditioning unit installation scene; The component identification step identifies one or more pressure gauges used during the installation of the air conditioning unit from the images acquired in the image acquisition step. The parameter identification step determines one or more parameters of the pressure gauge based on one or more features of the pressure gauge identified in the component identification step. The parameter comparison step compares the parameters determined in the parameter identification step with the parameters of the pressure gauges that should be used for the air conditioning unit model information in the pre-stored pressure gauge database to monitor the air conditioning unit installation process. The image acquisition step further includes: At the first moment, a third image relating to the installation scenario of the first air conditioning unit is acquired for the pressure gauge; At the second time, a fourth image relating to the installation scenario of the second air conditioning unit is acquired for the pressure gauge; The parameter identification step further includes: Based on one or more features of the pressure gauge in the third image, a first pressure value of the pressure gauge is determined; Based on one or more features of the pressure gauge in the fourth image, a second pressure value of the pressure gauge is determined; The change in the pressure value of the pressure gauge is determined based on the first pressure value, the second pressure value, the time point value of the first time, and the time point value of the second time.

2. The monitoring method for the installation process of an air conditioning unit as described in claim 1, characterized in that, The component identification step further includes: identifying one or more pressure vessels used during the installation of the air conditioning unit from the images acquired in the image acquisition step; The parameter identification step further includes: determining one or more parameters of the pressure vessel based on one or more features of the pressure vessel identified by the component identification step; The parameter comparison step further includes: comparing the parameters determined in the parameter identification step with the parameters of the pressure vessel to be used by the air conditioning unit in the pre-stored pressure vessel database, in order to monitor the air conditioning unit installation process.

3. The monitoring method for the installation process of an air conditioning unit as described in claim 2, characterized in that, The image acquisition step further includes: Acquire a first image of the first side of the pressure vessel; and / or Obtain a second image of the second side of the pressure vessel.

4. The monitoring method for the installation process of an air conditioning unit as described in claim 3, characterized in that, The first side is the side of the pressure vessel, and the features include at least one of the following: the curvature of the outline of the top of the pressure vessel, the diameter of the pressure vessel, the position or diameter of the pressure vessel inlet / outlet pipe, color, text, and height.

5. The monitoring method for the installation process of an air conditioning unit as described in claim 3, characterized in that, The second surface is the top surface of the pressure vessel, and the features include at least one of the following: the diameter of the pressure vessel, the position or diameter of the pressure vessel inlet / outlet pipe, color, and text.

6. The monitoring method for the installation process of an air conditioning unit as described in claim 1, characterized in that, The component identification step and / or the parameter identification step further include: An image dataset is provided, in which images have been labeled according to whether they contain relevant components, including pressure gauges, pressure vessels, wires, and circuit breaker units. Based on the image dataset, the classification model is trained to obtain the trained classification model; Based on the trained classification model, classify the images obtained in the image acquisition step.

7. The monitoring method for the installation process of an air conditioning unit as described in claim 6, characterized in that, The trained classification model classifies the images obtained in the image acquisition step based on the matching degree between the images obtained in the image acquisition step and the images in the image dataset.

8. The monitoring method for the installation process of an air conditioning unit as described in claim 6, characterized in that, It also includes: The images acquired in the image acquisition step are uploaded and saved in the image dataset, and, Based on the evaluation results, the images acquired in the image acquisition step are labeled to obtain an updated image dataset; Based on the updated image dataset, the trained classification model is further trained.

9. The monitoring method for the installation process of an air conditioning unit as described in claim 1, characterized in that, The first time point is the starting point for vacuuming the installed air conditioning unit.

10. The monitoring method for the installation process of an air conditioning unit as described in claim 1, characterized in that, The second time is the end time of vacuuming the installed air conditioning unit or the settling time after a specified period of time following the end of vacuuming.

11. The monitoring method for the installation process of an air conditioning unit as described in any one of claims 1, 9, and 10, characterized in that, The features include at least the range information of the pressure gauge and the pointer deflection angle information of the pressure gauge.

12. The monitoring method for the installation process of an air conditioning unit as described in claim 1, characterized in that, The component identification step further includes: identifying one or more electrical wires used during the installation of an air conditioning unit from the image acquired in the image acquisition step; The parameter identification step further includes: determining one or more parameters of the wire based on one or more features of the wire identified by the component identification step; The parameter comparison step further includes: comparing the parameters determined in the parameter identification step with the parameters of the wires that should be used by the air conditioning unit in the pre-stored wire database, in order to monitor the air conditioning unit installation process. The parameters include the diameter of the wire.

13. The monitoring method for the installation process of an air conditioning unit as described in claim 12, characterized in that, In the image acquisition step, an acquired image simultaneously contains image information of both the circuit breaker unit and the wire; The monitoring method for the installation process of the air conditioning unit also includes the following steps: Based on the image, determine the number of first pixels occupied by the circuit breaker unit in its width direction, and the number of second pixels occupied by the wire in its diameter direction; The diameter of the wire is determined based on the number of the first pixel and the number of the second pixel.

14. The monitoring method for the installation process of an air conditioning unit as described in claim 13, characterized in that, The wires include a first-color wire and a second-color wire; the images acquired in the image acquisition step include color images; and the monitoring method for the air conditioning unit installation process further includes the following steps: Process the color image to obtain a first color channel image and a second color channel image; Based on the first color channel image, determine the diameter of the first color wire; The diameter of the second-color wire is determined based on the second color channel image.

15. The monitoring method for the installation process of an air conditioning unit as described in claim 1, characterized in that, The feature is text information printed on the surface of related components, including pressure gauges, pressure vessels, wires, and circuit breaker units.

16. The monitoring method for the installation process of an air conditioning unit as described in claim 1, characterized in that, The monitoring method for the installation process of the air conditioning unit also includes the following steps: The comparison result sending step feeds back the comparison results obtained from the parameter comparison step to the installer or a pre-defined third party.

17. A computing device for monitoring the installation process of an air conditioning unit, characterized in that, include: At least one processor; as well as A memory, communicatively coupled to and readable by the at least one processor, storing processor-readable instructions that, when executed by the at least one processor, cause the at least one processor to perform the following operations: The image acquisition step involves acquiring one or more images of the air conditioning unit installation scene; The component identification step identifies one or more pressure gauges used during the installation of the air conditioning unit from the images acquired in the image acquisition step. The parameter identification step determines one or more parameters of the pressure gauge based on one or more features of the pressure gauge identified in the component identification step. The parameter comparison step compares the parameters determined in the parameter identification step with the parameters of the pressure gauges that should be used for the air conditioning unit model information in the pre-stored pressure gauge database to monitor the air conditioning unit installation process. The image acquisition step further includes: At the first moment, a third image relating to the installation scenario of the first air conditioning unit is acquired for the pressure gauge; At the second time, a fourth image relating to the installation scenario of the second air conditioning unit is acquired for the pressure gauge; The parameter identification step further includes: Based on one or more features of the pressure gauge in the third image, a first pressure value of the pressure gauge is determined; Based on one or more features of the pressure gauge in the fourth image, a second pressure value of the pressure gauge is determined; The change in the pressure value of the pressure gauge is determined based on the first pressure value, the second pressure value, the time point value of the first time, and the time point value of the second time.

18. A monitoring system for the installation process of an air conditioning unit, characterized in that, The monitoring system for the air conditioning unit installation process includes the computing device as described in claim 17 for monitoring the air conditioning unit installation process.

19. The monitoring system for the installation process of an air conditioning unit as described in claim 18, characterized in that, The monitoring system for the installation process of the air conditioning unit also includes a mobile terminal, which includes: An image acquisition device configured to acquire one or more images of an air conditioning unit installation scene; The wireless communication module is capable of uploading images acquired by the image acquisition device via wireless transmission and receiving the comparison results obtained from the parameter comparison step.

20. A device for monitoring the installation process of an air conditioning unit, characterized in that, The device includes: The image acquisition unit is used to acquire one or more images of the air conditioning unit installation scene; The component identification unit is used to identify one or more pressure gauges used during the installation of an air conditioning unit from the images acquired by the image acquisition unit. A parameter identification unit is used to determine one or more parameters of the pressure gauge based on one or more features of the pressure gauge identified by the component identification unit. The parameter comparison unit is used to compare the parameters determined by the parameter identification unit with the parameters of the pressure gauges that should be used for the model information of the air conditioning unit in the pre-stored pressure gauge database, so as to monitor the installation process of the air conditioning unit. The image acquisition unit is further configured to: At the first moment, a third image relating to the installation scenario of the first air conditioning unit is acquired for the pressure gauge; At the second time, a fourth image relating to the installation scenario of the second air conditioning unit is acquired for the pressure gauge; The parameter recognition unit is further used for: Based on one or more features of the pressure gauge in the third image, a first pressure value of the pressure gauge is determined; Based on one or more features of the pressure gauge in the fourth image, a second pressure value of the pressure gauge is determined; The change in the pressure value of the pressure gauge is determined based on the first pressure value, the second pressure value, the time point value of the first time, and the time point value of the second time.

21. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the monitoring method for the installation process of the air conditioning unit as described in any one of claims 1-16.

Images