A material position state detection system for implementing an intelligent workshop
The image acquisition and processing module enables intelligent detection of material status, solving the problem of low efficiency in manual inspection, realizing a fully automated intelligent workshop, improving efficiency and accuracy, and reducing labor costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- YOUZHU TECH (BEIJING) CO LTD
- Filing Date
- 2022-12-30
- Publication Date
- 2026-07-21
AI Technical Summary
Existing material level detection methods increase labor costs and are inefficient because they rely on manual observation to record material level status.
It employs an image acquisition module, an image processing module, and a data visualization module to achieve intelligent detection of material location status through image acquisition and processing, including image preprocessing, material location identification and numbering. Combined with data storage and intelligent transportation components, it realizes automated material transportation.
It enables intelligent detection of material status, improves work efficiency, reduces labor costs, realizes a fully automated intelligent workshop, has fast detection speed and high accuracy, saves labor costs, and the monitoring camera has multiple applications.
Smart Images
Figure CN116380193B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of smart workshop technology, and in particular to a material position status detection system for implementing a smart workshop. Background Technology
[0002] In recent years, the rapid development of the Internet of Things and big data has accelerated the pace of intelligent and information-based workshops. Intelligent workshops primarily utilize networks and software management systems to interconnect CNC automation equipment, enabling them to sense status, perform real-time data analysis, and thus achieve automated decision-making and precise command execution. Adopting intelligent workshops can significantly improve production efficiency and save on labor costs for enterprises.
[0003] The efficient scheduling of materials within a workshop is crucial for the smooth completion of processing. Intelligent transport vehicles can automate this process. However, controlling these vehicles typically relies on manual observation of images captured by monitoring equipment to record the occupancy status of each material location, thus controlling the vehicle's material release. Existing material location detection methods increase labor costs, and manually recording material location status results in low efficiency. Summary of the Invention
[0004] This specification provides one or more embodiments of a material position status detection system for implementing a smart workshop, which addresses the following technical problem: existing material position detection methods increase labor costs and record material position status manually, resulting in low work efficiency.
[0005] One or more embodiments of this specification employ the following technical solutions:
[0006] This specification provides one or more embodiments of a material location status detection system for implementing a smart workshop. The system includes: an image acquisition module, an image processing module, and a data visualization module. The image acquisition module is used to acquire real-time workshop images and send the real-time workshop images to the image processing module. The real-time workshop images include multiple material placement locations. The image processing module is used to perform image processing on the real-time workshop images to obtain material location information for each material placement location. The material location information includes material location status and material location data. The data visualization module is used to display the material location information for each material placement location.
[0007] Further, the image processing module is used to process the real-time workshop image to obtain material position information for each material placement position. Specifically, this includes: processing the real-time workshop image using a preset image processing method to obtain a processed binary workshop image; recognizing the processed binary workshop image using a preset method to obtain multiple material placement lines corresponding to multiple material placement positions in the binary workshop image; assigning material position numbers to the multiple material placement lines according to a preset numbering rule to determine the position information of the material placement position corresponding to each material placement line; determining the material detection area corresponding to each material placement position in the binary workshop image based on the multiple material placement lines; obtaining the grayscale value of each pixel in the material detection area in the binary workshop image; and determining whether there is material in the material detection area based on the grayscale value of each pixel to determine the material position status, wherein the material position status includes an occupied state and an idle state.
[0008] Furthermore, the image processing module processes the real-time workshop image using a preset image processing method to obtain a processed binary workshop image. Specifically, this includes: acquiring a pre-generated template detection box; cropping the real-time workshop image based on the template detection box to obtain a cropped workshop image; converting the cropped workshop image into a grayscale image to generate a cropped workshop grayscale image; performing Gaussian filtering on the cropped workshop grayscale image to obtain a processed cropped workshop grayscale image; binarizing the processed cropped workshop grayscale image to obtain a workshop binary image; and performing median filtering and dilation processing on the workshop binary image to obtain a processed workshop binary image.
[0009] Further, based on the grayscale value of each pixel, determining whether there is material in the material detection area to determine the material position state specifically includes: determining the number of pixels of a specified pixel in the material detection area according to the grayscale value of each pixel, wherein the grayscale value of the specified pixel satisfies a preset condition; if the number of pixels of the specified pixel is greater than or equal to a preset number threshold, it is determined that there is material in the material detection area, so that the material position state is determined to be a placeholder state; if the number of pixels of the specified pixel is less than the preset number threshold, it is determined that there is no material in the material detection area, so that the material position state is determined to be an idle state.
[0010] Furthermore, the processed workshop binary image is identified by a preset method to obtain multiple material placement lines corresponding to multiple material placement positions in the workshop binary image. Specifically, this includes performing a Hough transform on the processed workshop binary image to extract multiple material placement lines corresponding to multiple material placement positions in the workshop binary image.
[0011] Furthermore, according to a preset numbering rule, material position numbers are set for the multiple material placement lines to determine the location information of the material placement position corresponding to each material placement line. Specifically, this includes: obtaining the workshop numbering rule for the actual material placement position within the workshop; setting material position numbers for the multiple material placement lines according to the workshop numbering rule; and using the material position number corresponding to each material placement line as the location information of the material placement position corresponding to each material placement line.
[0012] Further, obtaining the pre-generated template detection box specifically includes: acquiring a workshop image of the workshop through the image acquisition module, wherein the workshop image includes a material placement area composed of multiple material placement positions; setting a template detection box in the workshop image according to a preset rule, wherein all material placement areas are within the frame selection area of the template detection box, and the area of the frame selection area is larger than the area of the material placement area; determining the position information of the template detection box in the workshop image, and storing the position information of the template detection box in a preset file, so as to obtain the template detection box according to the position information of the template detection box.
[0013] Furthermore, the system also includes a data storage module and an intelligent transportation component; wherein, the data storage module is used to store the material location information of each material placement location; the intelligent transportation component is used to call the storage module to obtain the material location information of each material placement location, and to perform material transportation based on the material location information of each material placement location.
[0014] Furthermore, the image acquisition module includes a video acquisition component and a control component; wherein, the control component is connected to the video acquisition component, and the control component is used to control the acquisition range of the video acquisition component; the video acquisition component is used to acquire the workshop monitoring video stream of the workshop, and obtain real-time workshop images based on the workshop monitoring video stream.
[0015] Furthermore, the image acquisition module is also used to calibrate the video acquisition component before the video acquisition component acquires the workshop monitoring video stream of the workshop.
[0016] The above-mentioned at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: Through the above technical solution, the status of the material position is intelligently detected, and the material is intelligently stored and retrieved according to the status of the material position, realizing intelligent material storage, which improves work efficiency in terms of time, improves the level of intelligence in terms of technology, and reduces personnel costs in terms of cost. The intelligent combination of the image acquisition module as the main module with other modules greatly improves work efficiency and realizes a fully automated intelligent workshop. In addition, the detection speed is fast and the detection accuracy is high, realizing intelligent production, saving labor costs, and the monitoring camera has multiple uses. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0018] Figure 1 This specification provides a schematic diagram of a material position status detection system for implementing a smart workshop, as shown in the embodiments of this specification.
[0019] Figure 2 This is a schematic diagram of another material position status detection system for implementing a smart workshop, provided as an embodiment of this specification. Detailed Implementation
[0020] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.
[0021] This specification provides an embodiment of a material position status detection system for implementing a smart workshop. Figure 1 This specification provides a schematic diagram of a material level status detection system for implementing a smart workshop, as illustrated in the embodiments of this specification. Figure 1As shown, the system includes an image acquisition module, an image processing module, and a data visualization module. The image acquisition module is used to acquire real-time images of the workshop and send them to the image processing module. The real-time workshop images include multiple material placement positions. The image processing module is used to process the real-time workshop images to obtain material position information for each material placement position. This material position information includes the material position status and position data. The data visualization module is used to display the material position information for each material placement position.
[0022] Furthermore, the image acquisition module includes a video acquisition component and a control component; wherein, the control component is connected to the video acquisition component, and the control component is used to control the acquisition range of the video acquisition component; the video acquisition component is used to acquire the workshop monitoring video stream of the workshop, and based on the workshop monitoring video stream, to obtain real-time workshop images.
[0023] In one embodiment of this specification, the image acquisition module includes a video acquisition component and a control component. The video acquisition component can be a surveillance camera, also known as a monitoring camera, and the control component can be a pan-tilt-zoom (PTZ) module. The surveillance camera is mounted directly above and in front of the material area to be detected, ensuring sufficient detection of the material area. The surveillance camera image acquisition module acquires image data through real-time transmission from the surveillance camera and processes each frame based on the image data. The PTZ module automatically adjusts according to the required observation position, resulting in a wider field of view for the surveillance camera and more comprehensive data acquisition.
[0024] It should be noted that the monitoring camera module achieves its functions through the installation location of the monitoring camera and the connection of the Python Software Development Kit (SDK). When installing the monitoring camera, it can mainly detect the material area. The connection of the Python SDK first requires installing the monitoring camera application, and then using the Python program to call the SDK connection module to control the monitoring camera.
[0025] In one embodiment of this specification, the monitoring system camera is first installed directly above and in front of the material detection area, at a height of 3-5 meters, suspended on the wall. The camera has 5 megapixels, supports pan-tilt rotation, automatic zoom, and exposure setting. The horizontal rotation is 360 degrees, and the vertical rotation is 180 degrees. It is hung directly above and in front of the material detection position, 3-5 meters above and in front of the camera. The monitoring system camera also has a secondary development module. Using Python, the camera can freely call the secondary development SDK module. This includes a series of program interfaces for pan-tilt rotation, adjusting camera brightness, exposure, and focus, enabling the monitoring head to be combined with Python to develop more intelligent functions, ultimately achieving autonomous detection and intelligent monitoring. The monitoring camera image data acquisition process includes: first, acquiring video data by reading the monitoring video stream in real time using the camera; and then acquiring each frame of the video stream. The monitoring camera's pan-tilt and other control modules control the pan-tilt rotation, zoom, exposure, and other settings to assist in image adjustment.
[0026] Furthermore, the image acquisition module is also used to calibrate the video acquisition component before it acquires the workshop monitoring video stream.
[0027] In one embodiment of this specification, the surveillance camera is calibrated before acquiring the surveillance video stream. The calibration steps include: first, calibrating the camera's distortion parameters, intrinsic parameter matrix, and extrinsic parameter matrix, and then adjusting the camera parameters to complete distortion correction. Specifically, the camera distortion correction steps are as follows: first, prepare a standard checkerboard pattern and take multiple photos from a suitable plane (ideally ten to twenty photos). Extract the checkerboard corner points from the photos, estimate the five intrinsic parameters and six extrinsic parameters under ideal conditions, estimate the distortion parameters under actual radial distortion using the least squares method, and improve the estimation accuracy using the maximum likelihood method. This completes the camera distortion correction. After completing the above calibration, image acquisition begins using the surveillance camera. Data acquisition is achieved by capturing the video stream in real time and then reading each frame of the video stream to collect data.
[0028] Specifically, the image processing module is used to process the real-time workshop image to obtain material location information for each material placement position. This includes: processing the real-time workshop image using a preset image processing method to obtain a processed binary image of the workshop; recognizing the processed binary image using a preset method to obtain multiple material placement lines corresponding to multiple material placement positions in the binary image; assigning material location numbers to the multiple material placement lines according to a preset numbering rule to determine the location information of the material placement position corresponding to each material placement line; determining the material detection area corresponding to each material placement position in the binary image based on the multiple material placement lines; obtaining the grayscale value of each pixel in the material detection area of the binary image; determining whether material exists in the material detection area based on the grayscale value of each pixel to determine the material location status, which includes an occupied status and an idle status.
[0029] Specifically, the image processing module processes the real-time workshop image using a preset image processing method to obtain a processed binary image of the workshop. This process includes: acquiring a pre-generated template detection box; cropping the real-time workshop image based on the template detection box to obtain a cropped workshop image; converting the cropped workshop image to grayscale to generate a cropped workshop grayscale image; performing Gaussian filtering on the cropped workshop grayscale image to obtain a processed cropped workshop grayscale image; binarizing the processed cropped workshop grayscale image to obtain a binary workshop image; and performing median filtering and dilation processing on the binary workshop image to obtain a processed binary workshop image.
[0030] Specifically, based on the grayscale value of each pixel, determining whether there is material in the material detection area to determine the material position status includes: determining the number of pixels of a specified pixel in the material detection area according to the grayscale value of each pixel, wherein the grayscale value of the specified pixel meets a preset condition; if the number of pixels of the specified pixel is greater than or equal to a preset number threshold, it is determined that there is material in the material detection area, and the material position status is determined to be a vacant state; if the number of pixels of the specified pixel is less than the preset number threshold, it is determined that there is no material in the material detection area, and the material position status is determined to be an idle state.
[0031] Specifically, the processed workshop binary image is identified using a preset method to obtain multiple material placement lines corresponding to multiple material placement positions in the workshop binary image. This includes performing a Hough transform on the processed workshop binary image to extract multiple material placement lines corresponding to multiple material placement positions in the workshop binary image.
[0032] Specifically, according to the preset numbering rules, material position numbers are set for the multiple material placement lines, and the location information of the material placement position corresponding to each material placement line is determined. This includes: obtaining the workshop numbering rules of the actual material placement position in the workshop; setting material position numbers for the multiple material placement lines according to the workshop numbering rules; and using the material position number corresponding to each material placement line as the location information of the material placement position corresponding to each material placement line.
[0033] Specifically, obtaining the pre-generated template detection box includes: acquiring a workshop image through the image acquisition module, wherein the workshop image includes a material placement area composed of multiple material placement positions; setting a template detection box in the workshop image according to a preset rule, wherein the material placement area is within the frame selection area of the template detection box, and the area of the frame selection area is larger than the area of the material placement area; determining the position information of the template detection box in the workshop image, and storing the position information of the template detection box in a preset file, so as to obtain the template detection box based on the position information of the template detection box.
[0034] In one embodiment of this specification, after calibration, image acquisition begins using a monitoring camera. Data acquisition is achieved by capturing a real-time video stream and reading each frame of the video stream. After data acquisition, Python is used to detect the status of the material location. This status falls into two categories: occupied and unoccupied, i.e., occupied and idle states. The coordinates are detected to guide the placement and retrieval of materials. The Python program is edited first, followed by editing the code that links to the monitoring camera, granting authorization, acquiring real-time video, adjusting the camera's position to the optimal location, and then taking and selecting images for data collection.
[0035] In one embodiment of this specification, after acquiring images captured by a surveillance camera, Python is used to read the images, and a calibration frame is manually set. It should be noted that the calibration frame should be referenced to the material placement line, and the calibration frame needs to encompass the material placement line; that is, the area of the calibration frame should be larger than the area of the material placement area. The size of the calibration frame can be customized. The coordinates of the upper left and lower right corners of the calibration frame are read as the coordinate information of the calibration frame, and the coordinate information of the calibration frame is stored in a preset file, which can be a txt file. The calibration frame can also be called a detection frame, template detection frame, etc. After completing the initial reading of the material area's location information, an initial detection template for the material area is created. After acquiring real-time workshop images, the initial detection template is used to process the real-time workshop images. This processing can include cropping to obtain the main image portion containing the material detection area. Using this initial template for image processing, rather than processing the entire image, reduces the computational load on the image.
[0036] In one embodiment of this specification, after obtaining the preliminary location information of the material area, a real-time connection is established with the monitoring camera to read the real-time monitoring video stream and process each frame of the image. First, the location information defined by the template is read, and the corresponding material placement line information is cropped out. By cropping, the image has material placement line information of the region of interest in the material area, and at the same time, the calculation of unnecessary information is reduced. Images that are not cropped will not be processed further.
[0037] In one embodiment of this specification, after obtaining the cropped image, each frame of the image is processed using the OpenCV Application Program Interface (API). First, the cropped image is converted into a grayscale image. The purpose of converting to grayscale is because color images contain too much information, while image recognition only requires information from grayscale images. Therefore, the purpose of image grayscale conversion is to improve the processing speed and enable faster calculations.
[0038] After obtaining the grayscale image, Gaussian filtering is performed. Gaussian filtering is a linear smoothing filter suitable for eliminating Gaussian noise. Simply put, Gaussian filtering is a process of weighted averaging of the image values. The value of each pixel is obtained by weighted averaging of its own value and the values of other pixels in its neighborhood. The specific operation of Gaussian filtering is as follows: a template (or convolution, mask) scans every pixel in the image, and the value of the center pixel of the template is replaced by the weighted average grayscale value of the pixels in the neighborhood defined by the template. In the embodiments of this specification, a 3x3 convolution kernel with a standard deviation of 1 is used to process the image.
[0039] After obtaining the image processed by Gaussian filtering, a binary image adaptive processing method was applied, with a segmentation calculation region size of 101 and a threshold set to 20. The purpose of binary images is to simplify the image, reduce data volume, and highlight the perceived target contours. Furthermore, processing and analyzing binary images allows for faster identification of target information. A binary image represents each pixel with only two possible values or grayscale levels. Typically, black and white, B&W, and monochrome images are used to represent binary images. Therefore, binary images only have two pixel values, "0" and "1". The drawback is that when representing images of people or landscapes, binary images can only display edge information, and the internal texture features are not clearly shown. In such cases, a grayscale image with richer texture features is used. This image only has black and white color, making the target contour values more prominent after processing.
[0040] After processing the binary image, the target outline becomes more prominent. Then, median filtering is applied to the image. Median filtering is a non-linear filter and also a statistical ranking filter. It sets the gray value of each pixel to the median of the gray values of all pixels within a certain neighborhood window. It has a good filtering effect on isolated noise pixels, salt-and-pepper noise, and impulse noise, preserving the edge characteristics of the image and preventing significant blurring. The role of median filtering in calculating the value of a point in a digital image or sequence is to sort the values of its neighbors from smallest to largest.
[0041] The median value is used to replace the isolated white spots of noise. After eliminating isolated, scattered white spots through median filtering, the image is dilated. First, a 3x3 convolution kernel is set, then dilation is applied to make the placement lines of the material area clearer and highlight the target line segments. The 3x3 dilation further emphasizes the target line segments. Finally, a Hough transform is applied to the image to extract the placement lines of the material area, thus determining their position.
[0042] After obtaining the location of the material placement line, the coordinates of the top-left corner (5) and bottom-right corner of the material placement line are obtained. These coordinates can be used to draw the corresponding material detection box in the real-time monitoring image. The workshop numbering rules for the actual material placement locations within the workshop are obtained, and material position numbers are assigned to multiple material placement lines according to these rules. The material position number corresponding to each material placement line is used as the location information of the material placement location for that line. For example, by iterating through...
[0043] Multiple detected material detection boxes are numbered from left to right, and their corresponding factory locations can also be numbered from left to right. Because the material locations detected in real-time monitoring are consistent with the factory material location numbers,
[0044] By monitoring and detecting the material boxes and their corresponding numbers in real time, the material location and number within the factory can be determined. Since the material location remains constant within the factory, knowing the corresponding number reveals its position within the factory. This provides the location information of the material location.
[0045] Furthermore, since the material locations detected in real-time are consistent with the factory's material location numbers, the accuracy of material placement can be determined by monitoring the occupancy status and location information of the detected material locations. For example, if a material should be placed in area number 1, but the detected occupancy status of area number 1 is vacant, it indicates an error in material placement, requiring immediate adjustment.
[0046] Based on the top-left and bottom-right corners of the corresponding material detection frame obtained above, the material detection frame can be obtained. After image processing within the material detection frame, white dots representing the outline are segmented. These white dots are obtained through the aforementioned steps. Because the final image processing step is binary image processing, the resulting image has only two grayscale values: 0 and 255. With only two values, if the material contains the corresponding material, the corresponding outline will be detected, and the outline will be represented by alternating black and white values. At this point, the white dots representing the material outline can be detected within the material frame. Therefore, the presence of material within the material frame can be determined by detecting these white dots. Furthermore, by counting the number of white dots within the material outline and applying a threshold, for example, a threshold of 3000 can be set. When the number of white dots is less than 3000, it indicates no material (idle state); when it is greater, it indicates material (occupying a space). This makes the detection more flexible and accurate.
[0047] Furthermore, the system also includes a data storage module and an intelligent transportation component; wherein, the data storage module is used to store the material location information of each material placement position; the intelligent transportation component is used to call the storage module to obtain the material location information of each material placement position, and to perform material transportation based on the material location information of each material placement position.
[0048] In one embodiment of this specification, the intelligent transportation component can be an Automated Guided Vehicle (AGV), specifically an indoor vision-guided transportation vehicle. The data storage module may include an SQL database. SQL is a database language with multiple functions such as data manipulation and data definition. This language is interactive and provides great convenience to users. The database management system should fully utilize SQL to improve the working quality and efficiency of the computer application system. In this embodiment, the SQL data has storage, interactive communication capabilities. The data storage module can also be called a data storage and communication connection module. Python connects to the database for storage and retrieval. The program obtains the coordinates and classification of empty spaces from the test images and stores them in the SQL database.
[0049] A PC connects to a server to build an SQL database, which stores the current status of the material storage area, i.e., the material location information for each material placement position. It also handles communication and retrieval functions. The PC connects to the server and stores and retrieves data from the SQL database. The program uses test images to determine the coordinates and classification of empty material locations, storing this information in the SQL database and corresponding to the characteristic information of each material location. This information is then retrieved and processed by other programs. For example, an AGV (Automated Guided Vehicle) is used to control the AGV to transport materials to available material locations.
[0050] In one embodiment of this specification, after obtaining the material location status information, the material location information for each material placement location is displayed through a data visualization module. This data visualization module, also known as a program data visualization module, is installed on a PC and used to display the current status of the material area in real time, enabling intelligent planning of AGVs. The visualization in the program data visualization module is programmed using Python and QT5, with a corresponding framework and display interface. The PC displays the real-time detected status of the material area, and this status is combined with big data analysis, ultimately displayed on the visualization platform, showing the available material locations, the number of empty locations, the position number that is empty, and the percentage of available locations. This allows for better display of the real-time status of materials to workers, enabling more rational configuration of material delivery. Simultaneously, the monitoring cameras and some program adjustment threshold parameters can be debugged, further simplifying and improving the operation of the program.
[0051] The above technical solution enables intelligent detection of material position status, and intelligent storage and retrieval of materials based on the material position status, achieving intelligent material storage. This improves work efficiency in terms of time, enhances the level of intelligence in terms of technology, and reduces personnel costs. The image acquisition module, as the main component, is intelligently integrated with other modules, greatly improving work efficiency and realizing a fully automated intelligent workshop. In addition, the detection speed is fast and the detection accuracy is high, enabling intelligent production, saving labor costs, and making the monitoring camera multi-functional.
[0052] This specification also provides another material position status detection system for implementing a smart workshop. Figure 2 This is a schematic diagram of another material level status detection system for implementing a smart workshop, provided in an embodiment of this specification. Figure 2 As shown, the system includes: a monitoring system camera module, a monitoring camera image acquisition and control pan-tilt module, a main program system detection module, a data storage and communication connection module, and a program data visualization module.
[0053] The monitoring system's camera module is installed directly above and in front of the material area being monitored, ensuring comprehensive detection of the material zone. The camera's image acquisition module captures image data via real-time transmission from the camera and processes each frame according to the program. The pan-tilt-zoom (PTZ) module automatically adjusts the camera's position based on the required observation location, resulting in a wider field of view and more comprehensive data acquisition.
[0054] The main program system's detection module processes each image frame in a loop, using the corresponding image processing program on the PC to detect the real-time dynamics within each frame, ultimately determining the number and specific empty slots in the material area. The image manipulation within the main program system's processing module achieves this by minimizing code complexity, resulting in faster processing and clearer visualization.
[0055] The data storage and communication module connects to the server via a PC to build an SQL database. This database stores the current status of the material area and also handles communication and retrieval functions. Through the PC-server connection, it stores and retrieves data from the SQL database. The program analyzes test images to determine the coordinates and classification of empty spaces, storing this information in the SQL database. This information corresponds to the characteristic information of each material location, which is then retrieved and processed by other programs. Finally, it controls the AGV (Automated Guided Vehicle) to transport materials.
[0056] The program data visualization module, installed on a PC, displays the real-time status of the material storage area for intelligent AGV (Automated Guided Vehicle) planning. The visualization module is programmed using Python and QT5, with a corresponding framework and display interface. It displays the real-time detected status of the material storage area on the PC, combining this data with big data analytics to ultimately display it on a big data visualization platform. This shows the available material locations, the number of empty locations, the position of the empty location, and the percentage of available locations. This provides workers with a better view of the real-time material status, enabling more rational material delivery configuration. Simultaneously, it allows for the adjustment of monitoring cameras and program threshold parameters, further simplifying and enhancing the program's operation.
[0057] This specification also provides a method for detecting the status of material locations in a smart workshop. The method includes: saving an image of the corresponding material placement location; drawing a corresponding detection box on the required detection image; saving the coordinates of the detection box; connecting to a monitoring camera; and processing each frame of the real-time video to determine the material placement status. The process of processing each frame of the real-time video to determine the material placement status is as follows: first, reading the coordinate values of the template detection box; then, cropping the real-time image according to the detection box area corresponding to the template detection box coordinate values; converting the cropped image into a grayscale image; then processing the image using Gaussian filtering, binary filtering, and median filtering; setting a 3x3 convolution kernel; then applying dilation to the image; and finally using Hough transform to detect straight lines, partially selecting them to obtain the corresponding material line coordinates; reading the corresponding selected box position and then segmenting it to obtain the number of white dots within each material line; and judging by a threshold value for the number of white dots set for the material. If the number is higher than the threshold, it indicates that the material location is occupied, and the material location status is "occupied"; if it is lower than the threshold, the material location is not occupied, and the material location status is "idle". Ultimately, it can determine whether the material placement position is empty, how many positions are empty, and which position is empty.
[0058] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0059] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.
[0060] The devices, media, and methods provided in the embodiments of this specification are one-to-one correspondences. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.
[0061] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0062] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0065] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0066] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0067] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0068] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0069] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.
Claims
1. A material position status detection system for realizing a smart workshop, characterized in that, The system includes: an image acquisition module, an image processing module, and a data visualization module; The image acquisition module is used to acquire real-time workshop images and send the real-time workshop images to the image processing module, wherein the real-time workshop images include multiple material placement positions; The image processing module is used to process the real-time workshop image to obtain material location information for each material placement location, wherein the material location information includes material location status and material location data. The data visualization module is used to display the material location information for each material placement position; The image processing module is used to process the real-time workshop image to obtain material location information for each material placement position, specifically including: The real-time workshop image is processed using a preset image processing method to obtain a processed binary image of the workshop. The processed workshop binary image is identified by a preset method to obtain multiple material placement lines corresponding to multiple material placement positions in the workshop binary image; According to the preset numbering rules, material position numbers are set for the multiple material placement lines, and the position information of the material placement position corresponding to each material placement line is determined; Based on the multiple material placement lines, the material detection area corresponding to each material placement position is determined in the workshop binary image; Obtain the grayscale value of each pixel in the material detection area of the workshop binary image; Based on the grayscale value of each pixel, it is determined whether there is material in the material detection area, so as to determine the material position state, wherein the material position state includes occupied state and idle state; The image processing module processes the real-time workshop image using a preset image processing method to obtain a processed binary image of the workshop, specifically including: Obtain the pre-generated template detection box; Based on the template detection box, the real-time workshop image is cropped to obtain a cropped workshop image; The cutting workshop image is converted into a grayscale image to generate a grayscale image of the cutting workshop; The grayscale image of the cutting workshop is subjected to Gaussian filtering to obtain the processed grayscale image of the cutting workshop; The processed grayscale image of the cutting workshop is binarized to obtain a binary image of the workshop. The binary image of the workshop is then subjected to median filtering and dilation to obtain a processed binary image of the workshop. The system also includes a data storage module and intelligent transportation components; The data storage module is used to store the material location information for each material placement location; The intelligent transportation component is used to call the storage module to obtain the material location information of each material placement position, and to perform material transportation based on the material location information of each material placement position.
2. The material position status detection system for realizing a smart workshop according to claim 1, characterized in that, Based on the grayscale value of each pixel, determining whether material exists in the material detection area to determine the material position status specifically includes: Based on the grayscale value of each pixel, the number of pixels of a specified pixel is determined in the material detection area, wherein the grayscale value of the specified pixel satisfies a preset condition; If the number of pixels of the specified pixel point is greater than or equal to a preset number threshold, it is determined that there is material in the material detection area, so that the material position status is determined to be an occupied state. If the number of pixels at the specified pixel point is less than a preset threshold, it is determined that there is no material in the material detection area, so that the material position status is determined to be idle.
3. The material position status detection system for realizing a smart workshop according to claim 1, characterized in that, The processed workshop binary image is identified using a preset method to obtain multiple material placement lines corresponding to multiple material placement positions in the workshop binary image, specifically including: A Hough transform is performed on the processed workshop binary image to extract multiple material placement lines corresponding to multiple material placement positions in the workshop binary image.
4. A material position status detection system for realizing a smart workshop according to claim 1, characterized in that, According to a preset numbering rule, material position numbers are assigned to the multiple material placement lines, and the location information of the material placement position corresponding to each material placement line is determined, specifically including: Obtain the workshop numbering rules for the actual material placement locations within the workshop; According to the workshop numbering rules, material location numbers are assigned to the multiple material placement lines; The material position number corresponding to each material placement line is used as the location information of the material placement position corresponding to each material placement line.
5. A material position status detection system for realizing a smart workshop according to claim 1, characterized in that, Obtain the pre-generated template detection box, specifically including: The image acquisition module acquires images of the workshop, including a material placement area consisting of multiple material placement positions. According to preset rules, a template detection frame is set in the workshop image, wherein the material placement area is within the frame selection area of the template detection frame, and the area of the frame selection area is larger than the area of the material placement area. The position information of the template detection box in the workshop image is determined, and the position information of the template detection box is stored in a preset file so that the template detection box can be obtained according to the position information of the template detection box.
6. A material position status detection system for realizing a smart workshop according to claim 1, characterized in that, The image acquisition module includes a video acquisition component and a control component; The control component is connected to the video acquisition component, and the control component is used to control the acquisition range of the video acquisition component; The video acquisition component is used to acquire the workshop monitoring video stream and obtain real-time workshop images based on the workshop monitoring video stream.
7. A material position status detection system for realizing a smart workshop according to claim 6, characterized in that, The image acquisition module is also used to calibrate the video acquisition component before the video acquisition component acquires the workshop monitoring video stream of the workshop.