A battery box appearance detection method and device

Abstract

The present invention discloses a method and device for detecting the appearance of a battery box, which comprises: step S1, collecting a large number of target foreign body images, using a deep learning algorithm to construct and train a deep neural network model; step S2, packaging the constructed deep neural network model to generate an executable file; step S3, constructing a traditional machine vision framework to run as the main software, and separately establishing a new thread to call the executable file to start running; step S4, creating a server when the executable file is started, connecting the main software to the server as a client, so as to realize data communication between the traditional machine vision framework and the server executable file; step S5, when performing the appearance detection of the battery box, the traditional machine vision framework as the main software collects and obtains the image of the appearance of the battery box and sends it to the server, executes the executable file on the server to obtain the recognition result, and returns the recognition result to the traditional machine vision framework for output.

Description

Technical Field

[0001] The present invention relates to a battery box appearance detection method and device, and in particular to a battery box appearance detection method and device combining traditional industrial vision with a deep learning algorithm. Background Art

[0002] At present, the appearance inspection of electric vehicle battery boxes in the automotive industry mainly relies on manual visual inspection or traditional visual inspection of specific targets in specific areas. It cannot effectively cover all parts and accurately judge various and multi-form foreign objects for the detection of foreign objects with high randomness. At this time, it is necessary to combine deep learning algorithms with traditional industrial visual inspection methods.

[0003] Python is a high-level programming language that has great advantages over other languages ​​in algorithm implementation. For example, to complete the same task, C language requires 1,000 lines of code, C# requires 100 lines of code, and Python only requires 20 lines of code. Python also has many open source function libraries, so deep learning algorithms based on Python have been widely used.

[0004] For industrial visual inspection methods that combine deep learning algorithms to achieve visual inspection, the current common methods include visual software frameworks developed based on C# language combined with deep learning algorithms based on Python. It can use C++ programs to call python files, and then make them into dynamic link libraries (dll), and call this dll file in C#. However, this method runs too slowly, and a large number of operating environments and dependencies need to be configured to successfully call dll files, resulting in low scalability; and if the C++ language is used to rebuild the framework, the workload is large. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a battery box appearance detection method and device, by which the main software calls a Python-based deep learning algorithm, so that the main software and the Python-based deep learning algorithm can run separately, and the stable data transmission between the Python-based deep learning algorithm and the main software can be achieved through Socket communication. The Python-based deep learning algorithm can be integrated into the existing traditional machine vision detection framework, thereby achieving the purpose of foreign body detection on the appearance of the battery box by combining traditional industrial vision with deep learning algorithms, and at the same time omitting the tedious process of C++ encapsulating dynamic link libraries.

[0006] To achieve the above object, the present invention provides a battery box appearance detection method, comprising the following steps:

[0007] Step S1, by collecting a large number of target foreign body images, using a deep learning algorithm to build and train a deep neural network model to complete the visual recognition task;

[0008] Step S2, packaging the constructed deep neural network model to generate an executable file;

[0009] Step S3, constructing a traditional machine vision framework to run as the main software, and separately establishing a new thread to call the executable file to start running;

[0010] Step S4, creating a server when the executable file is started, connecting the main software as a client to the server, so as to realize data communication between the main software of the traditional machine vision framework and the executable file of the server;

[0011] Step S5, when performing battery box appearance inspection, the traditional machine vision framework as the main software collects and obtains the image of the battery box appearance and sends it to the server, executes the executable file on the server to obtain the recognition result, and returns the recognition result to the traditional machine vision framework as the main software for output.

[0012] Preferably, in step S1, the deep neural network model is constructed based on a deep learning algorithm in Python language, and the constructed model is trained using the collected target foreign body images.

[0013] Preferably, in step S2, the constructed deep neural network model such as the yolov5 model is packaged using Python's third-party library Pyinstaller, and the executable file is finally generated to ensure that the executable file can be run even in an environment where no software is installed after packaging.

[0014] Preferably, the interface input variables of the packaged deep neural network model are pictures captured by an industrial camera, and the interface output variables are the type, quantity and coordinate value information of foreign objects identified in the picture.

[0015] Preferably, in step S3, a traditional machine vision framework is constructed based on C# language, and the traditional machine vision framework is run as the main software. After the traditional machine vision framework is run as the main software, a new thread is separately established to call the executable file, so that the two programs run separately.

[0016] Preferably, the main software creates a new thread through the Process class to call the executable file to start and run it.

[0017] Preferably, in step S4, when the executable file is started, a server for Socket communication with the main software is created, and a Socket connection is established with the main software.

[0018] Preferably, data communication between the main software and the server-side executable file is achieved through write function and read function in Socket communication.

[0019] Preferably, in step S5, after the main software acquires the appearance image through image acquisition, it sends the image address to the server executable file through the write function. After reading the image, the server executes the executable file, thereby identifying it through the deep neural network model, and obtaining the identification results such as the type, quantity and coordinate value of the foreign matter, and sending the identification results to the main software. The main software reads the information through the read function and displays the foreign matter area.

[0020] In order to achieve the above object, the present invention also provides a battery box appearance detection device, comprising:

[0021] A visual recognition model building unit is used to collect a large number of target foreign body images, use deep learning algorithms to build and train a deep neural network model, and complete visual recognition tasks;

[0022] An executable file generation unit, used to package the constructed deep neural network model and generate an executable file;

[0023] A traditional machine vision framework construction unit is used to construct a traditional machine vision framework to run as the main software, and to separately create a new thread to call the executable file to start running;

[0024] A server creation unit, used to create a server when the executable file is started, link the main software as a client with the server, and realize data communication between the main software and the server;

[0025] The appearance detection processing unit is used to control the traditional machine vision framework as the main software to collect and obtain the image of the battery box appearance and send it to the server when performing battery box appearance detection, execute the executable file on the server to obtain the recognition result, and return the recognition result to the traditional machine vision framework as the main software for output and display.

[0026] Compared with the prior art, a battery box appearance detection method and device of the present invention uses the main software to call the executable file python.exe to enable the two programs to run separately, and realizes stable data transmission between the deep learning algorithm and the main software through Socket communication, thereby integrating the deep learning algorithm into the existing traditional machine vision detection framework, achieving the purpose of foreign body detection on the appearance of the battery box by combining traditional industrial vision with the deep learning algorithm, and at the same time omitting the cumbersomeness of C++ encapsulation of dynamic link libraries. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of a method for detecting the appearance of a battery box according to the present invention;

[0028] Figure 2 A system structure diagram of a battery box appearance inspection device of the present invention;

[0029] Figure 3 Schematic diagram of the process of battery box appearance inspection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following describes the implementation of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0031] Figure 1 FIG. 1 is a flow chart of the steps of a battery box appearance detection method of the present invention. Figure 1 As shown, the present invention provides a battery box appearance detection method, comprising the following steps:

[0032] Step S1, by collecting a large number of target foreign body images, using a deep learning algorithm to build and train a deep neural network model to complete the visual recognition task.

[0033] In a specific embodiment of the present invention, a large number of target foreign body images are collected, and a deep neural network model, namely, a yolov5 model, is constructed based on the Python language. The model is trained by using the large number of collected target foreign body images to achieve accurate detection of target foreign bodies in the image through the yolov5 model. Since the detection of targets in images based on the Python-based deep learning algorithm is an existing mature technology, the present invention will not elaborate on it.

[0034] Step S2: package the constructed deep neural network model to generate an executable file.

[0035] In a specific embodiment of the present invention, Python's third-party library Pyinstaller is used to package the constructed deep neural network model, such as the yolov5 model. During packaging, the third-party library Pyinstaller will automatically download and install modules and interpreters such as matplotlib, numpy, scipy, etc. required for running the python code, and finally generate an executable file python.exe to ensure that the executable file can run even on a computer without the software running environment installed after packaging.

[0036] In a specific embodiment of the present invention, the packaged deep neural network model has the following characteristics: first, the interface input variable is the picture captured by the industrial camera; second, the interface output variable is information such as the type, quantity and coordinate value of the foreign matter identified in the picture.

[0037] Step S3, construct a traditional machine vision framework to run as the main software, and separately create a new thread to call the executable file to start running.

[0038] In a specific embodiment of the present invention, an existing traditional machine vision framework is developed based on the C# language, and the traditional machine vision framework is run as the main software. After the traditional machine vision framework is run as the main software, a new thread is established separately to call the executable file python.exe, so that the two programs can run separately. In a specific embodiment of the present invention, the main software of the traditional machine vision framework can establish a new thread through the Process class to call the executable file python.exe, so that it can be started and run. Since the development of the existing traditional machine vision framework based on the C# language is already a prior art, it will not be repeated here.

[0039] Step S4, creating a server when the executable file is started, connecting the main software as a client to the server to achieve data communication between the main software of the traditional machine vision framework and the executable file of the server.

[0040] In a specific embodiment of the present invention, when the executable file python.exe is started, a server for Socket communication with the main software is created, and then the main software of the traditional machine vision framework is used as a client to link with the executable file python.exe as the server, and data communication between the main software of the traditional machine vision framework and the server executable file python.exe is realized through the write function and read function in the Socket communication.

[0041] Step S5, when performing battery box appearance inspection, the traditional machine vision framework as the main software collects and obtains the image of the battery box appearance and sends it to the server, executes the executable file on the server to obtain the recognition result, and returns the recognition result to the traditional machine vision framework as the main software for output and display.

[0042] Specifically, when performing appearance inspection of the battery box, a camera device is used to capture the appearance image of the battery box. After the main software of the traditional machine vision framework acquires the appearance image through image acquisition, the image address is sent to the server python.exe through the write function. After reading the image, the server executes the executable file python.exe, and can identify it through the deep neural network model yolov5 model, and obtain identification results such as the type, quantity and coordinate values ​​of the foreign matter, and send the type, quantity and coordinate values ​​of the foreign matter to the main software of the traditional machine vision framework. The main software of the traditional machine vision framework reads the information through the read function and displays the foreign matter area, thereby realizing the embedding of the Python-based deep learning algorithm into the traditional machine vision framework developed based on the C# language.

[0043] Figure 2 This is a system structure diagram of a battery box appearance inspection device of the present invention. Figure 2 As shown, the present invention provides a battery box appearance detection device, comprising:

[0044] The visual recognition model building unit 201 is used to collect a large number of target foreign body images and use a deep learning algorithm to build and train a deep neural network model to complete the visual recognition task.

[0045] In a specific embodiment of the present invention, the visual recognition model building unit 201 collects a large number of target foreign body images, constructs a neural network based on the Python language, namely, the yolov5 model, and trains with the large number of collected target foreign body images to achieve accurate detection of target foreign bodies in the image through the yolov5 model. Since the detection of targets in images based on Python's deep learning algorithm is an existing mature technology, the present invention will not elaborate on it.

[0046] The executable file generating unit 202 is used to package the constructed deep neural network model to generate an executable file.

[0047] In a specific embodiment of the present invention, the executable file generation unit 202 uses Python's third-party library Pyinstaller to package the constructed deep neural network model, such as the yolov5 model. During packaging, the third-party library Pyinstaller will automatically download and install modules and interpreters such as matplotlib, numpy, scipy, etc. required for running the python code, and finally generate an executable file python.exe to ensure that the packaged executable file can run even on a computer that does not have a software running environment installed.

[0048] In a specific embodiment of the present invention, the packaged deep neural network model has the following characteristics: first, the interface input variable is the picture captured by the industrial camera; second, the interface output variable is information such as the type, quantity and coordinate value of the foreign matter identified in the picture.

[0049] The traditional machine vision framework construction unit 203 is used to construct the existing traditional machine vision framework to run as the main software, and to separately create a new thread to call the executable file to start running.

[0050] In a specific embodiment of the present invention, the traditional machine vision framework construction unit 203 develops an existing traditional machine vision framework based on the C# language, and runs the traditional machine vision framework as the main software. After the main software runs, a new thread is established separately to call the executable file python.exe, so that the two programs run separately. In a specific embodiment of the present invention, the main software of the traditional machine vision framework can establish a new thread through the Process class to call the executable file python.exe to start and run it. Since the development of the existing traditional machine vision framework based on the C# language is already a prior art, it will not be repeated here.

[0051] The server creation unit 204 is used to create a server when the executable file is started, link the main software as a client with the server, and realize data communication between the main software and the server.

[0052] In a specific embodiment of the present invention, when the executable file python.exe is started, a server for Socket communication is created through the server creation unit 204, and then the main software of the traditional machine vision framework is used as a client to link with python.exe as a server, and data communication between the main software and the server executable file python.exe is realized through the write and read functions in the Socket.

[0053] The appearance detection processing unit 205 is used to control the traditional machine vision framework as the main software to collect and obtain the image of the battery box appearance and send it to the server when performing battery box appearance detection, execute the executable file on the server to obtain the recognition result, and return the recognition result to the traditional machine vision framework as the main software for output and display.

[0054] Specifically, when performing appearance inspection of the battery box, a camera device is used to capture the appearance image of the battery box. After the main software of the traditional machine vision framework acquires the appearance image through image acquisition, the image address is sent to the server python.exe through the write function. After reading the image, the server executes the executable file python.exe, and can identify it through the deep neural network model yolov5 model, and obtain identification results such as the type, quantity and coordinate values ​​of the foreign matter, and send the type, quantity and coordinate values ​​of the foreign matter to the main software of the traditional machine vision framework. The main software of the traditional machine vision framework reads the information through the read function and displays the foreign matter area, thereby realizing the embedding of the Python-based deep learning algorithm into the traditional machine vision framework developed based on the C# language.

[0055] Example

[0056] Figure 3 This is a schematic diagram of the appearance inspection of a battery box by combining traditional industrial vision with a deep learning algorithm according to an embodiment of the present invention. Figure 3 As shown, the present invention provides a battery box appearance inspection method combining traditional industrial vision and deep learning algorithm, and the process is as follows:

[0057] Step 1: Collect a large number of target foreign body images and build a yolov5 model based on Python language to achieve accurate detection of foreign bodies in images; use Python's third-party library Pyinstaller to package the yolov5 model. During packaging, Pyinstaller will automatically download and install modules and interpreters such as matplotlib, numpy, scipy, etc. required for running python code, and finally produce an executable file python.exe to ensure that it can run even on computers without software running environment installed after packaging; when the existing traditional machine vision framework developed based on C# language starts running, python.exe is called through the Process class to start and run it.

[0058] The packaged model has the following features: first, the interface input variable is the image captured by the industrial camera; second, the interface output variable is the type, quantity, coordinate value and other information of the foreign objects identified in the image;

[0059] Step 2, the executable file python.exe is started and creates the server for Socket communication;

[0060] Step 3, the main software acts as a client to link with the server python.exe;

[0061] Step 4: The link is established, and the server sends "1" to confirm the link. The data communication between the main software and python.exe can be achieved through the write and read functions in the Socket.

[0062] Step 5, the main software collects images and sends the image address to python.exe through the write function;

[0063] Step 6, after reading the image, the server python.exe feeds back "2" to the main software and starts to identify it through the yolov5 model. After the identification is completed, the identification results such as the type, quantity and coordinate value of the foreign matter are sent to the main software of the existing framework;

[0064] Step 7, the main software receives the recognition result, returns "3", reads the information through the read function, and then performs the foreign body area display operation, so that the deep learning algorithm in the yolov5 model can be embedded into the existing framework;

[0065] Step 8: After receiving "3", the server python.exe clears the result cache and returns "4";

[0066] Step 9, after receiving "4", the main software sends "0";

[0067] Step 10: When the server python.exe receives "0", the link is disconnected.

[0068] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Any person skilled in the art may modify and alter the above embodiments without violating the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be as set forth in the claims.

Claims

1. A battery box appearance detection method, comprising the following steps: Step S1, by collecting a large number of target foreign body images, using a deep learning algorithm to build and train a deep neural network model to complete the visual recognition task; Step S2, packaging the constructed deep neural network model to generate an executable file; wherein, Use Python's third-party library Pyinstaller to package the constructed deep neural network model and finally generate the executable file to ensure that the executable file can run even in an environment where no software is installed after packaging; Step S3, constructing a traditional machine vision framework to run as the main software, and separately establishing a new thread to call the executable file to start running; wherein, the traditional machine vision framework is constructed based on the C# language, the traditional machine vision framework is run as the main software, and after the traditional machine vision framework is run as the main software, a new thread is separately established to call the executable file, so that the two programs run separately; Step S4, creating a server when the executable file is started, connecting the main software as a client to the server to achieve data communication between the main software of the traditional machine vision framework and the server executable file; Step S5, when performing battery box appearance inspection, the traditional machine vision framework as the main software collects and obtains the image of the battery box appearance and sends it to the server, executes the executable file on the server to obtain the recognition result, and returns the recognition result to the traditional machine vision framework as the main software for output.

2. A battery box appearance detection method as claimed in claim 1, characterized in that In step S1, the deep neural network model is constructed based on the deep learning algorithm of Python language, and the constructed model is trained using the collected target foreign body images.

3. A battery box appearance detection method as claimed in claim 1, characterized in that : The interface input variables of the packaged deep neural network model are the pictures collected by the industrial camera, and the interface output variables are the types, quantities and coordinate value information of the foreign objects identified in the pictures.

4. A battery box appearance detection method as claimed in claim 1, characterized in that : The main software creates a new thread through the Process class to call the executable file to start and run it.

5. A battery box appearance detection method as claimed in claim 1, characterized in that In step S4, when the executable file is started, a server for Socket communication with the main software is created, and a Socket connection is established with the main software.

6. A battery box appearance detection method as claimed in claim 5, characterized in that :Data communication between the main software and the server executable file is realized through the write function and read function in Socket communication.

7. A battery box appearance detection method as claimed in claim 6, characterized in that : In step S5, after the main software acquires the image of the appearance through image acquisition, it sends the image address to the server executable file through the write function. After reading the image, the server executes the executable file, thereby identifying it through the deep neural network model and obtaining the recognition result. The recognition result includes the type, quantity and coordinate value of the foreign matter, and sends the recognition result to the main software. The main software reads the recognition result through the read function and displays the foreign matter area.

8. A battery box appearance detection device, comprising: A visual recognition model building unit is used to collect a large number of target foreign body images, use deep learning algorithms to build and train a deep neural network model, and complete visual recognition tasks; The executable file generation unit is used to package the constructed deep neural network model and generate an executable file; wherein, Use Python's third-party library Pyinstaller to package the constructed deep neural network model and finally generate the executable file to ensure that the executable file can run even in an environment where no software is installed after packaging; A traditional machine vision framework construction unit, used to construct a traditional machine vision framework to run as the main software, and to separately create a new thread to call the executable file to start running; wherein, the traditional machine vision framework is constructed based on the C# language, the traditional machine vision framework is run as the main software, and after the traditional machine vision framework is run as the main software, a new thread is separately created to call the executable file, so that the two programs run separately; A server creation unit, used to create a server when the executable file is started, link the main software as a client with the server, and realize data communication between the main software and the server; The appearance detection processing unit is used to control the traditional machine vision framework as the main software to collect and obtain the image of the battery box appearance and send it to the server when performing battery box appearance detection, execute the executable file on the server to obtain the recognition result, and return the recognition result to the traditional machine vision framework as the main software for output and display.

Images