Grabbing information extraction system based on computer vision

Abstract

The invention relates to the technical field of computer vision, in particular to a captured information extraction system based on computer vision, which comprises an image real-time acquisition module for acquiring frame images and environmental data in a monitoring period in real time, a product data acquisition module for acquiring form data of a processed product, an image data analysis module, a data processing module and a data processing module, the contrast analysis module is used for carrying out contrast analysis on frame images in a monitoring period and carrying out light processing on the frame images, and the product position analysis module is used for extracting product morphological characteristics, identifying processed products in the frame images, analyzing product position information according to an identification result and storing the product position information. The displacement analysis module is used for analyzing the displacement abnormity of the processed product, and the grabbed information extraction module is used for analyzing the grabbed information and outputting the grabbed information to a user. According to the method, the extraction efficiency of the captured information is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a capture information extraction system based on computer vision. Background Art

[0002] Grasping is a basic and important task in industrial automation and robotics. Traditional grasping systems usually rely on pre-programmed paths and fixed positions, which have great limitations when facing complex and dynamic environments. In recent years, with the development of computer vision technology, vision-based grasping information extraction systems have gradually become a research hotspot. These systems can perceive the environment in real time and extract information such as the position, posture and shape of the target object, thereby realizing flexible and intelligent grasping operations. However, existing computer vision-based grasping systems still have some problems, such as low detection accuracy, slow processing speed, and poor adaptability to environmental changes. Therefore, it is of great significance to develop an efficient, accurate and adaptable grasping information extraction system.

[0003] Chinese Patent Publication No. CN115439529A discloses a positioning method and system based on a color deformable material image positioning algorithm, which includes a computer, an industrial robot, a camera, a gripping device, a transparent material tray and a surface light source, wherein the camera is used to obtain a color deformable material image; the computer obtains a color deformable material image HSV model based on the color deformable material image, judges the material color based on the color deformable material image HSV model, judges the material area based on the grayscale threshold of the color deformable material image HSV model, obtains the center point and inclination angle of the material based on the material area, and sends the positioning information reflecting the center point and inclination angle of the material to the industrial robot and the gripping device; it can be seen that the invention only analyzes the gripping mechanical energy at a fixed position, and does not conduct a detailed analysis of the extraction of dynamic gripping information in the intelligent production process, and cannot achieve efficient gripping information extraction, and there is a problem of low efficiency in the extraction of gripping information. Summary of the invention

[0004] The object of the present invention is to provide a computer vision-based crawling information extraction system to solve at least one of the problems existing in the prior art.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A computer vision-based crawling information extraction system, characterized by comprising:

[0007] The real-time image acquisition module is used to acquire frame images and environmental data within the monitoring period in real time;

[0008] Product data acquisition module, used to obtain the morphological data of processed products;

[0009] An image data analysis module is used to perform contrast analysis on frame images within a monitoring period, and to perform light processing on each frame image according to the contrast analysis results of each frame image within the monitoring period, and to adjust the contrast analysis process of the frame image according to the environmental data within the monitoring period;

[0010] The product position analysis module is used to extract product morphological features according to the morphological data of the processed products, and to identify the processed products in each frame image according to the product morphological features and the frame images after light processing during the monitoring period, and to analyze the product position information according to the recognition results and store the product position information;

[0011] A displacement analysis module is used to analyze the displacement anomaly of the processed product based on the stored product position information;

[0012] The grab information extraction module is used to analyze the grab information according to the displacement abnormality analysis results of the processed products and the product position information, and output the grab information to the user.

[0013] Further, the image data analysis module includes a contrast analysis unit, which is used to calculate the contrast α(i) of each frame image within the monitoring period;

[0014] The contrast analysis unit compares the contrast α(i) of each frame image with the preset contrast A, and performs contrast analysis on each frame image according to the comparison result, and the contrast analysis result includes normal contrast and abnormal contrast.

[0015] Furthermore, the image data analysis module also includes an ambient light analysis unit, which is used to compare the average ambient light intensity g during the monitoring period with each preset light intensity to determine the ambient light abnormality, wherein the ambient light abnormality includes weak ambient light, normal ambient light and strong ambient light, and when the ambient light is weak, the preset contrast is adjusted to A', and when the ambient light is strong, the preset contrast is adjusted to A".

[0016] Furthermore, the image data analysis module further includes a shadow analysis unit, which calculates the shadow similarity β(i) of each frame image;

[0017] The shadow analysis unit is used to compare the shadow similarity with the preset similarity B to determine the abnormality of the environmental shadow, which includes normal and abnormal. When the environmental shadow is abnormal, the first preset environmental light intensity is optimized to G1'.

[0018] Furthermore, the image data analysis module further includes a light processing unit, which is used to perform light processing on each frame image according to the contrast analysis result of each frame image in the monitoring period: if the contrast of the frame image is abnormal, the light processing unit sets the gray value of each pixel in the frame image to t(i, j); if the contrast of the frame image is normal, the light processing unit does not change the gray value of the pixel in the frame image;

[0019] Wherein, t(i,j) represents the gray value of the j-th pixel in the i-th frame image after light processing.

[0020] Furthermore, the product position analysis module includes a morphological feature extraction unit, which is used to perform numerical matrixing on the morphological image of the processed product to obtain a morphological matrix Q;

[0021] The morphological feature extraction unit is used to denoise the morphological matrix Q: if Q(k)<YQ, the morphological feature extraction unit sets the gray value of the pixel to 255; if Q(k)≥YQ, the morphological feature extraction unit does not set the gray value of the pixel; wherein YQ is a preset binary gray threshold;

[0022] The morphological feature extraction unit uses the denoised morphological matrix Q as the product morphological feature.

[0023] Furthermore, the product position analysis module also includes a feature recognition unit, which is used to identify the processed products in each frame image according to the product morphological characteristics and the frame images after light processing within the monitoring period. If the recognition is successful, the feature recognition unit records the coordinate point ZB(i,f) of the processed product in each frame image after light processing, and records the recognition morphological matrix SQ(i,f) of the successful recognition. If the recognition fails, no record is made; wherein SQ(i,f) represents the recognition morphological matrix of the fth successful recognition of the i-th frame image.

[0024] Furthermore, the product position analysis module further includes a position marking unit, and the position marking unit is used to mark different recognition form matrices in the same frame image as different marked products;

[0025] The position marking unit performs secondary marking on each marked product: if ZB(i,f1)+T / μ×V=ZB(i+1,f2), the position marking unit marks the f2th marked product in the i+1th frame image as the f1th marked product in the i-th frame image, and uses ZB(i+1,f2) as the position sequence of the f1th marked product in the i-th frame image; if ZB(i,f1)+T / μ×V≠ZB(i+1,f2), the position marking unit does not perform secondary marking; wherein V is the moving speed of the conveyor belt;

[0026] The position marking unit counts the position sequence of each marked product and stores it as product position information.

[0027] Furthermore, the displacement analysis module is used to count the number of marked products NM and the number of marked products N1 in the first frame image, and calculate the ratio of offset products η;

[0028] The displacement analysis module analyzes the displacement anomaly of the processed product according to the displacement product ratio η, and the displacement anomaly of the processed product includes normal and abnormal.

[0029] Further, the grabbing information extraction module includes a grabbing information extraction unit, which is used to extract the grabbing information according to the displacement abnormality analysis result of the processed product and the product position information: if the displacement abnormality of the processed product is normal, the grabbing information extraction unit sets the predicted grabbing time of each marked product to YT and sets the predicted grabbing point coordinates to YZB; if the displacement abnormality of the processed product is abnormal, the grabbing information extraction unit does not set the predicted grabbing time and the predicted grabbing point coordinates, and sends the marked product to the user as an offset product; ZB(h) represents the coordinates of the h-th marked product, dis(h) represents the longitudinal distance of the h-th marked product from the grabbing point, and h represents the number of the marked product;

[0030] The grabbing information extraction unit uses the predicted grabbing point coordinates and predicted grabbing time of each marked product as the grabbing information.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows: by processing the collected frame images one by one, efficient exposure of the processed products can be achieved, and then by identifying and marking the processed products and analyzing the product position information based on the marking results, accurate positioning and tracking of the processed products can be achieved, and then by analyzing the abnormal displacement state, accurate extraction of grasping information can be achieved, thereby improving the extraction efficiency of grasping information. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 Schematic diagram of the structure of the computer vision-based crawling information extraction system of this embodiment.

[0034] Figure 2 FIG. 4 is a schematic diagram of the structure of the image data analysis module of this embodiment.

[0035] Figure 3 This is a schematic diagram of the structure of the product location analysis module of this embodiment.

[0036] Figure 4 Schematic diagram of the structure of the crawling information extraction module of this embodiment. DETAILED DESCRIPTION

[0037] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present invention.

[0038] It should be noted that, although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0039] Specifically, the computer vision-based grasping information extraction system described in this embodiment is applied to the extraction of product grasping information in the industrial intelligent production process; the industrial intelligent production process described in this embodiment is the grasping process of products on a conveyor belt; the product described in this embodiment is specifically a steel product with a smooth surface, and its shape is input by the user; the industrial intelligent production described in this embodiment is industrial production under a complex lighting environment.

[0040] See also Figure 1 As shown, it is a structural diagram of the computer vision-based crawling information extraction system described in this embodiment, including:

[0041] The real-time image acquisition module is used to acquire frame images and environmental data within the monitoring period.

[0042] Specifically, the frame image in this embodiment is a grayscale image, and its grayscale value range is 0-255; this embodiment does not specifically limit the frame image acquisition method, and technical personnel in this field can set it freely, as long as the frame image acquisition requirements are met. The frame image described in this embodiment is a continuous image within a period of time taken by a fixed camera at a fixed frequency, and the moving direction of the conveyor belt in this embodiment is parallel to one side of the frame image; it can be understood that the fixed frequency described in this embodiment is set by technical personnel in this field, and the fixed frequency in this embodiment is 30 frames / second; the environmental data includes an environmental light analysis unit, and the environmental data is obtained by a light sensor.

[0043] Specifically, this embodiment does not impose any specific limitation on the value of the duration of the monitoring period, and those skilled in the art can freely set it as long as the value of the duration of the monitoring period is satisfied. In this embodiment, the value of the duration of the monitoring period is set to 1.5 seconds.

[0044] Please continue reading Figure 1 As shown, the system further includes a product data acquisition module, which is used to acquire the morphological data of the processed product.

[0045] Specifically, the morphological data of the processed product includes a morphological image of the processed product; the morphological image of the processed product is a proportionally reduced image of the processed product; the present embodiment does not specifically limit the method for obtaining the morphological data of the processed product, and those skilled in the art can freely set it as long as the requirements for obtaining the morphological data of the processed product are met. The present embodiment obtains it through user interactive input; at the same time, the morphological image of the processed product described in the present embodiment is the same as the frame image in size, proportion and format.

[0046] Please continue reading Figure 1 As shown, the system also includes:

[0047] An image data analysis module is connected to the image real-time acquisition module, and is used for performing light processing on frame images within a monitoring period.

[0048] See also Figure 2 As shown, the image data analysis module includes a contrast analysis unit, which is used to perform contrast analysis on the frame images within the monitoring period, so as to pre-process the collected frame images to highlight the processed product morphology of the frame images;

[0049] The contrast analysis unit calculates the contrast α(i) of each frame image within the monitoring period and sets

[0050] Where i = 1, 2...N, N is the number of frame images collected in the monitoring period, N = μ × T, μ is the acquisition frequency of the frame image, T is the duration of the monitoring period, b(i) is the grayscale average of the i-th frame image, M is the number of pixels in the frame image, and Grey(i,r) represents the grayscale value of the r-th pixel of the i-th frame image;

[0051] The contrast analysis unit compares the contrast of each frame image with the preset contrast A, and performs contrast analysis on each frame image according to the comparison result: if α(i)<A, the contrast analysis unit determines that the contrast of the frame image is abnormal; if α(i)≥A, the contrast analysis unit determines that the contrast of the frame image is normal; by analyzing the image contrast, the accuracy of the light processing of the image is improved, thereby improving the accuracy of the product position analysis.

[0052] Specifically, this embodiment does not impose any specific limitation on the value of the preset contrast A, and only needs to meet the value requirement of the preset contrast A. It is worth noting that since the processed product described in this embodiment is a steel product with a smooth surface, its preset contrast is higher than the preset contrast of the processed product with a rough surface. In this embodiment, the optimal value of the preset contrast A is 80.

[0053] Please continue reading Figure 2 As shown, the image data analysis module also includes an ambient light analysis unit, which is connected to the contrast analysis unit, and is used to analyze the ambient light abnormality according to the ambient light intensity within the monitoring period to eliminate the influence of the ambient light on the contrast analysis of the frame image;

[0054] The ambient light analysis unit is used to compare the average ambient light intensity g during the monitoring period with each preset light intensity, and analyze the ambient light abnormality according to the comparison result: if g<G1, the ambient light analysis unit determines that the ambient light during the monitoring period is weak, and adjusts the preset contrast to A', setting A'=A×[1-(G1-g) / G1]; if G1≤g<G2, the ambient light analysis unit determines that the ambient light during the monitoring period is normal; if g≥G2, the ambient light analysis unit determines that the ambient light during the monitoring period is strong, and adjusts the preset contrast to A", setting A"=A×[1+(G1-g) / G1];

[0055] Among them, G1 is the first preset ambient light intensity, G2 is the second preset ambient light intensity, and G1<G2; by adjusting the influence of ambient light on frame image shooting, the influence of ambient light on steel products with smooth surfaces is amplified and adjusted, thereby improving the accuracy of image light processing, and thus improving the accuracy of product position analysis.

[0056] Specifically, this embodiment does not impose any specific limitations on the values ​​of the first preset ambient light intensity G1 and the second preset ambient light intensity G2. Those skilled in the art can set them freely as long as the value requirements of the first preset ambient light intensity G1 and the second preset ambient light intensity G2 are met. In this embodiment, the first preset ambient light intensity G1 can be set to 50l x, and the second preset ambient light intensity G2 can be set to 100l x.

[0057] Please continue reading Figure 2 As shown, the image data analysis module further includes a shadow analysis unit, which is connected to the ambient light analysis unit, and is used to analyze the abnormality of the ambient shadow according to the ambient shadow grayscale value within the monitoring period, so as to eliminate the influence of the ambient shadow on the recognition of the processed product;

[0058] The shadow analysis unit is used to calculate the shadow similarity β(i) of each frame image, and set β(i)=b(i) / y, where y is the gray value of the non-processed product area of ​​the environment during the monitoring period;

[0059] The shadow analysis unit is used to compare the shadow similarity with the preset similarity B, and analyze the abnormality of the environmental shadow according to the comparison result, and optimize the contrast analysis process of the frame image according to the analysis result: if β(i)<B, the shadow analysis unit determines that the environmental shadow of the i-th frame image is normal and does not perform optimization; if β(i)≥B, the shadow analysis unit determines that the environmental shadow of the i-th frame image is abnormal, and optimizes the first preset environmental light intensity to G1', setting G1'=ln{e-[β(i)-B] / B}; wherein e is the natural logarithm; by analyzing the influence of the shadow on the grayscale image, it is determined whether the shadow pixel is too close to the grayscale value of the frame image, and then the influence of the environmental light is optimized, thereby improving the accuracy of the light processing of the image, and then improving the accuracy of the product position analysis.

[0060] Specifically, this embodiment does not impose any specific limitation on the value of the preset similarity B, and those skilled in the art can freely set it as long as the value requirement of the preset similarity B is met. In this embodiment, the preset similarity B can be set to 0.6.

[0061] Please continue reading Figure 2As shown, the image data analysis module also includes a light processing unit, which is connected to the contrast analysis unit, and is used to perform light processing on each frame image according to the contrast analysis result of each frame image in the monitoring period, so as to realize the deep exposure of the processed product: if the contrast of the frame image is abnormal, the light processing unit sets the gray value of each pixel in the frame image to t(i,j), and sets t(i,j)=pt(i,j)×[pt(i,j)-127] / 127×exp{[A-α(i)] / A}; if the contrast of the frame image is normal, the light processing unit does not change the gray value of the pixel in the frame image;

[0062] Among them, pt(i,j) represents the original grayscale value of the j-th pixel in the i-th frame image, and t(i,j) represents the grayscale value of the j-th pixel in the i-th frame image after light processing; by performing multi-valued processing on the image, the exposure of the frame image is improved, thereby improving the accuracy of the light processing of the image, and thereby improving the accuracy of the product position analysis.

[0063] Please continue reading Figure 1 As shown, the system also includes a product position analysis module, which is connected to the image data analysis module and the product data acquisition module. The product position analysis module is used to match image product features according to the frame image after light processing within a monitoring period and the morphological data of the processed product, and to analyze the product position information according to the matching results, and to store the product position information.

[0064] See also Figure 3 As shown, the product position analysis module includes a morphological feature extraction unit, and the morphological feature extraction unit is used to extract product morphological features according to the morphological data of the processed product;

[0065] The morphological feature extraction unit performs numerical matrixing on the morphological image of the processed product to obtain a morphological matrix Q;

[0066] The morphological feature extraction unit is used to denoise the morphological matrix Q: if Q(k)<YQ, the morphological feature extraction unit sets the gray value of the pixel to 255; if Q(k)≥YQ, the morphological feature extraction unit does not set the gray value of the pixel; wherein YQ is a preset binary gray threshold;

[0067] The morphological feature extraction unit uses the denoised morphological matrix Q as the product morphological feature; through the digitization and denoising process of the product morphological feature, the accurate extraction of the product morphology is achieved, the accuracy of the product position information analysis is improved, and then the accuracy and efficiency of the capture information extraction are improved.

[0068] Specifically, this embodiment does not impose any specific limitation on the value of the preset binary grayscale threshold YQ. Those skilled in the art can freely set it as long as the value requirement of the preset binary grayscale threshold YQ is met. In this embodiment, the optimal value of YQ is 225.

[0069] Specifically, the process of "converting the morphological image of the processed product into a numerical matrix" in this embodiment is: using the gray value of each pixel in the morphological image of the processed product as the value of each element of the matrix, and the matrix elements correspond to the pixels of the morphological image one by one;

[0070] Please continue reading Figure 3 As shown, the product location analysis module also includes a feature recognition unit, which is connected to the morphological feature extraction unit. The feature recognition unit is used to identify the processed products in each frame image according to the product morphological features and the frame images after light processing within the monitoring period. If the recognition is successful, the feature recognition unit records the coordinate point ZB(i,f) of the processed product in each frame image after light processing, and records the recognition morphological matrix SQ(i,f) of the successful recognition. If the recognition fails, no record is made; wherein SQ(i,f) represents the recognition morphological matrix of the fth successful recognition of the i-th frame image; by identifying the product and storing the product coordinates, data support is provided for the continuous analysis process of multiple images, thereby improving the accuracy of the product location information analysis.

[0071] Specifically, this embodiment does not specifically limit the process of "identifying the processed products in each frame image based on the product morphological characteristics and the frame images after light processing within the monitoring period". Technical personnel in this field can set it freely as long as their process requirements are met. In this embodiment, the recognition process can be implemented through OpenCV; at the same time, in this embodiment, "the coordinate points of the processed products in each frame image after light processing" are specifically the image coordinate points of the center points of the processed products in the frame images after each light processing; in this embodiment, the analysis method of the recognition morphological matrix is ​​the same as that of the morphological matrix Q, which will not be repeated in this embodiment.

[0072] It can be understood that the process of "identifying the processed products in each frame image" described in this embodiment does not take into account the difference in position or direction, and the recognition is successful when the product shape in the frame image matches the product shape feature.

[0073] Please continue reading Figure 3 As shown, the product position analysis module further includes a position marking unit, which is connected to the feature recognition unit and is used to mark the processed product in each frame image according to the product position information;

[0074] The position marking unit marks different recognition form matrices in the same frame image as different marking products;

[0075] The position marking unit performs secondary marking on each marked product: if ZB(i,f1)+T / μ×V=ZB(i+1,f2), the position marking unit marks the f2th marked product in the i+1th frame image as the f1th marked product in the i-th frame image, and uses ZB(i+1,f2) as the position sequence of the f1th marked product in the i-th frame image; if ZB(i,f1)+T / μ×V≠ZB(i+1,f2), the position marking unit does not perform secondary marking; wherein V is the moving speed of the conveyor belt;

[0076] The position marking unit counts the position sequence of each marked product and stores it as product position information; by merging the product identification results for the same product, continuous product positioning is achieved, and then product movement trajectory analysis is achieved, thereby improving the accuracy of product position analysis.

[0077] Specifically, the marking process described in this embodiment is an ascending numbering process; meanwhile, "T / μ×V" is the displacement distance in the coordinate system.

[0078] Please continue reading Figure 1 As shown, the system further includes a displacement analysis module, which is connected to the product position analysis module, and is used to analyze the displacement abnormality of the processed product according to the stored product position information;

[0079] The displacement analysis module counts the number of marked products NM and the number of marked products N1 in the first frame image, and calculates the ratio of offset products η, setting η=N1 / NM;

[0080] The displacement analysis module analyzes the displacement abnormality of the processed product according to the offset product ratio η: if η<U, the displacement analysis module determines that the displacement abnormality of the processed product within the monitoring period is abnormal; if η≥U, the displacement analysis module determines that the displacement abnormality of the processed product within the monitoring period is normal; wherein U is a preset offset product ratio; by analyzing the displacement abnormality of the product in the frame image within the monitoring period to determine whether the displacement of the product can be predicted, the accuracy and efficiency of the capture information extraction are improved.

[0081] Specifically, this embodiment does not impose any specific limitation on the value of the preset offset product ratio U, and those skilled in the art can freely set it as long as the value requirement of the preset offset product ratio U is met. For example, the preset offset product ratio U can be set to 0.9.

[0082] Please continue reading Figure 1As shown, the system also includes a grasping information extraction module, which is connected to the displacement analysis module. The grasping information analysis module is used to extract the grasping information according to the displacement abnormality analysis results of the processed products within the monitoring period and the product position information, and output the grasping information to the user.

[0083] See also Figure 4 As shown, the grabbing information extraction module includes a grabbing information extraction unit, which is used to extract the grabbing information according to the displacement abnormality analysis result of the processed product and the product position information: if the displacement abnormality of the processed product is normal, the grabbing information extraction unit sets the predicted grabbing time of each marked product to YT, sets the predicted grabbing point coordinates to YZB, sets YT=dis(h) / V, YZB=ZB(h)+dis(h); if the displacement abnormality of the processed product is abnormal, the grabbing information extraction unit does not set the predicted grabbing time and the predicted grabbing point coordinates, and sends the marked product to the user as an offset product; ZB(h) represents the coordinates of the hth marked product, dis(h) represents the longitudinal distance of the hth marked product from the grabbing point, and h represents the number of the marked product;

[0084] The grabbing information extraction unit uses the predicted grabbing point coordinates and predicted grabbing time of each marked product as grabbing information; by setting the predicted grabbing point coordinates and predicted grabbing time, accurate prediction of the movement trajectory of regular products is achieved, thereby improving the accuracy and efficiency of grabbing information extraction.

[0085] Specifically, the calculation logic of "YZB=ZB(h)+dis(h)" in this embodiment is to add dis(h) to the value of the coordinate axis along the direction of advancement of the conveyor belt in the coordinate ZB(h) of the h-th marked product, that is, the data format of ZB(h) is (x, y). If the coordinate axis along the direction of advancement of the conveyor belt is the y-axis, the calculation result is (x, y+dis(h)).

[0086] Please continue reading Figure 4 As shown, the captured information extraction module also includes an output unit, which is connected to the captured information extraction unit, and the output unit is used to output the captured information to the user.

[0087] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A computer vision-based crawling information extraction system, characterized in that: include: The real-time image acquisition module is used to acquire frame images and environmental data within the monitoring period in real time; Product data acquisition module, used to obtain the morphological data of processed products; An image data analysis module is used to perform contrast analysis on frame images within a monitoring period, and to perform light processing on each frame image according to the contrast analysis results of each frame image within the monitoring period, and to adjust the contrast analysis process of the frame image according to the environmental data within the monitoring period; The product position analysis module is used to extract product morphological features according to the morphological data of the processed products, and to identify the processed products in each frame image according to the product morphological features and the frame images after light processing during the monitoring period, and to analyze the product position information according to the recognition results and store the product position information; A displacement analysis module is used to analyze the displacement anomaly of the processed product based on the stored product position information; The grab information extraction module is used to analyze the grab information according to the displacement abnormality analysis results of the processed products and the product position information, and output the grab information to the user.

2. The computer vision-based crawling information extraction system according to claim 1, characterized in that: The image data analysis module includes a contrast analysis unit, which is used to calculate the contrast α(i) of each frame image within the monitoring period; The contrast analysis unit compares the contrast α(i) of each frame image with the preset contrast A, and performs contrast analysis on each frame image according to the comparison result, and the contrast analysis result includes normal contrast and abnormal contrast.

3. The computer vision-based crawling information extraction system according to claim 2, characterized in that: The image data analysis module also includes an ambient light analysis unit, which is used to compare the average ambient light intensity g during the monitoring period with each preset light intensity to determine the ambient light abnormality, which includes weak ambient light, normal ambient light and strong ambient light. When the ambient light is weak, the preset contrast is adjusted to A', and when the ambient light is strong, the preset contrast is adjusted to A".

4. The computer vision-based crawling information extraction system according to claim 3, characterized in that: The image data analysis module further includes a shadow analysis unit, which calculates the shadow similarity β(i) of each frame image; The shadow analysis unit is used to compare the shadow similarity with the preset similarity B to determine the abnormality of the environmental shadow, which includes normal and abnormal. When the environmental shadow is abnormal, the first preset environmental light intensity is optimized to G1'.

5. The computer vision-based crawling information extraction system according to claim 4, characterized in that: The image data analysis module further includes a light processing unit, which is used to perform light processing on each frame image according to the contrast analysis result of each frame image in the monitoring period: if the contrast of the frame image is abnormal, the light processing unit sets the gray value of each pixel in the frame image to t(i, j); if the contrast of the frame image is normal, the light processing unit does not change the gray value of the pixel in the frame image; Wherein, t(i,j) represents the gray value of the j-th pixel in the i-th frame image after light processing.

6. The computer vision-based crawling information extraction system according to claim 5, characterized in that: The product position analysis module includes a morphological feature extraction unit, which is used to perform numerical matrixing on the morphological image of the processed product to obtain a morphological matrix Q; The morphological feature extraction unit is used to denoise the morphological matrix Q: if Q(k)<YQ, the morphological feature extraction unit sets the gray value of the pixel to 255; if Q(k)≥YQ, the morphological feature extraction unit does not set the gray value of the pixel; wherein YQ is a preset binary gray threshold; The morphological feature extraction unit uses the denoised morphological matrix Q as the product morphological feature.

7. The computer vision-based crawling information extraction system according to claim 6, characterized in that: The product position analysis module also includes a feature recognition unit, which is used to identify the processed products in each frame image according to the product morphological features and the frame images after light processing within the monitoring period. If the recognition is successful, the feature recognition unit records the coordinate point ZB(i,f) of the processed product in each frame image after light processing, and records the recognition morphological matrix SQ(i,f) of the successful recognition. If the recognition fails, no record is made; wherein SQ(i,f) represents the recognition morphological matrix of the fth successful recognition of the i-th frame image.

8. The computer vision-based crawling information extraction system according to claim 7, characterized in that: The product position analysis module further includes a position marking unit, which is used to mark different recognition form matrices in the same frame image as different marked products; The position marking unit performs secondary marking on each marked product: if ZB(i,f1)+T / μ×V=ZB(i+1,f2), the position marking unit marks the f2th marked product in the i+1th frame image as the f1th marked product in the i-th frame image, and uses ZB(i+1,f2) as the position sequence of the f1th marked product in the i-th frame image; if ZB(i,f1)+T / μ×V≠ZB(i+1,f2), the position marking unit does not perform secondary marking; wherein V is the moving speed of the conveyor belt; The position marking unit counts the position sequence of each marked product and stores it as product position information.

9. The computer vision-based crawling information extraction system according to claim 8, characterized in that: The displacement analysis module is used to count the number of marked products NM and the number of marked products N1 in the first frame image, and calculate the ratio of offset products η; The displacement analysis module analyzes the displacement anomaly of the processed product according to the displacement product ratio η, and the displacement anomaly of the processed product includes normal and abnormal.

10. The computer vision-based crawling information extraction system according to claim 9, characterized in that: The grabbing information extraction module includes a grabbing information extraction unit, which is used to extract the grabbing information according to the displacement abnormality analysis result of the processed product and the product position information: if the displacement abnormality of the processed product is normal, the grabbing information extraction unit sets the predicted grabbing time of each marked product to YT and sets the predicted grabbing point coordinates to YZB; if the displacement abnormality of the processed product is abnormal, the grabbing information extraction unit does not set the predicted grabbing time and the predicted grabbing point coordinates, and sends the marked product to the user as an offset product; ZB(h) represents the coordinates of the h-th marked product, dis(h) represents the longitudinal distance of the h-th marked product from the grabbing point, and h represents the number of the marked product; The grabbing information extraction unit uses the predicted grabbing point coordinates and predicted grabbing time of each marked product as the grabbing information.

Images