Data transmission method and system of industrial vision cloud service platform

Abstract

The present application relates to the technical field of edge data transmission, and in particular to a data transmission method and system of an industrial vision cloud service platform. The present application analyzes the frame picture blur condition of the inspected workpiece in the quality inspection process and the modeling quality inspection duration, evaluates the historical computing power resource condition of the cloud platform, serves as a reference for evaluating the computing power resource condition of the current cloud platform, then dynamically adjusts the frame picture compression rate in combination with the frame picture blur condition of the workpiece to be inspected in the quality inspection process, so as to reduce the transmission bandwidth, simultaneously consider the working condition of the cloud server in real time, improve the edge data transmission effect, and further improve the subsequent quality inspection effect.

Description

Technical Field

[0001] This invention relates to the field of edge data transmission technology, and specifically to a data transmission method and system for an industrial vision cloud service platform. Background Technology

[0002] Industrial vision is a technology that uses computer vision to analyze and process images collected during industrial production processes. It is widely used in production line inspection, quality control, and automated operation. However, industrial vision quality inspection applications generate a large amount of inspection data, and the computing power of edge computing may not be sufficient for real-time processing. Therefore, edge inspection data is usually compressed and transmitted to a cloud service platform for storage and processing. This not only improves quality inspection efficiency but also allows for data storage and sharing.

[0003] However, the computing capacity of cloud servers is dynamic. An excessively high compression ratio will consume a large amount of decompression resources. When the computing capacity of the cloud server is low, it may lead to a decrease in the performance stability of the cloud service platform and may also affect the accuracy of the detection data. When the computing capacity of the cloud server is high, an excessively low compression ratio may lead to a waste of transmission bandwidth and may also lead to excessively long transmission time or transmission failure, thereby affecting the quality inspection efficiency. Therefore, choosing an appropriate compression ratio is crucial. An inappropriate compression ratio will seriously affect the transmission effect of detection data at the edge. Summary of the Invention

[0004] To address the technical problem of poor data transmission performance for edge monitoring in existing technologies, the present invention aims to provide a data transmission method and system for an industrial vision cloud service platform. The specific technical solution adopted is as follows:

[0005] A data transmission method for an industrial vision cloud service platform, the method comprising:

[0006] Acquire video frame sequences of each workpiece from different inspection angles during the quality inspection process, wherein the workpieces include the current workpiece to be inspected and the inspected workpieces; acquire the modeling and inspection time of each inspected workpiece on the cloud platform;

[0007] In each video frame sequence, different frame images are matched and tracked, and the relative change parameters of each frame image are obtained based on the difference of matching pixels between adjacent frame images; during the quality inspection process of each workpiece, the frame image blur coefficient under each frame image number is obtained based on the relative change parameters of the frame images under the same frame image number in different video frame sequences.

[0008] During the quality inspection process of each inspected workpiece, cloud performance parameters of the cloud platform are constructed based on the frame blur coefficients under all frame sequence numbers and the modeling and detection duration; during the quality inspection process of the current workpiece to be inspected, the compression rate of each frame in each video frame sequence is obtained based on the frame blur coefficients under each frame sequence number and the preset compression rate range, combined with the cloud performance parameters corresponding to the quality inspection processes of all inspected workpieces;

[0009] The corresponding frame image is compressed according to the compression ratio, and the compression result is transmitted to the cloud platform.

[0010] Furthermore, the method for obtaining the relative change parameter includes:

[0011] In each video frame sequence, a Cartesian coordinate system is constructed with the center of each frame as the origin to obtain the position coordinates of each pixel in each frame. Between each frame and the previous adjacent frame, the Euclidean distance between the position coordinates of each pair of matched pixels is used as a variation sub-parameter, and the mean of the variation sub-parameters between all matched pixels is used as the relative variation parameter between each frame and the previous adjacent frame.

[0012] Furthermore, the method for obtaining the frame blur coefficient includes:

[0013] During the quality inspection process of each workpiece, between every two video frame sequences, the absolute value of the difference between the relative change parameters between frames with the same frame number is used as the fuzzy sub-parameter for the corresponding frame number.

[0014] Under each frame number, the frame blur coefficient is obtained by combining the blur sub-parameters between all two different video frame sequences during the workpiece quality inspection process.

[0015] Furthermore, the method for obtaining the cloud performance parameters includes:

[0016] During the quality inspection of each inspected workpiece, the negative correlation mapping result of the sum of the frame fuzziness coefficients under all frame sequence numbers is used as the first performance parameter, and the negative correlation mapping result of the modeling and inspection time is used as the second performance parameter.

[0017] The first performance parameter and the second performance parameter are used as vector elements to construct a two-dimensional vector, which is then used as the cloud performance parameter of the cloud platform during the quality inspection process of the corresponding known workpiece.

[0018] Furthermore, the method for obtaining the compression ratio includes:

[0019] By integrating the first performance parameter and the second performance parameter from the cloud performance parameters of the cloud platform during the quality inspection process of each inspected workpiece, the computational redundancy parameter of the cloud platform in each quality inspection process is obtained; based on the computational redundancy parameter in the quality inspection process of all inspected workpieces, the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected is obtained.

[0020] By integrating the negative correlation mapping result of the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected with the frame blur coefficient under each frame number, the frame compression weight under each frame number in all video frame sequences of the current workpiece to be inspected is obtained.

[0021] Based on the frame compression weight and the preset compression rate range, the compression rate of each frame in each video frame sequence during the current quality inspection process is obtained.

[0022] Furthermore, the method for obtaining the cloud computing capacity coefficient includes:

[0023] The computational redundancy parameters of the cloud platform in the previous adjacent quality inspection process of the current workpiece to be inspected are used as the current computational redundancy parameters; based on the fluctuation deviation of the computational redundancy parameters relative to the average level in the quality inspection process of each inspected workpiece, the computational redundancy stable parameters are obtained.

[0024] By combining the current computational redundancy parameter and the computational redundancy stability parameter, the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected is obtained.

[0025] Furthermore, the method for obtaining the compression ratio of each frame in each video frame sequence during the current quality inspection process based on the frame compression weight and the preset compression ratio range includes:

[0026] The left endpoint of the preset compression rate range is used as the base compression rate. The length of the preset compression rate range is weighted using the frame compression weight, and the weighted result is used as the adjusted compression rate. The base compression rate is added to the adjusted compression rate to obtain the compression rate of the frame at the corresponding frame number.

[0027] Furthermore, methods for matching and tracking different frames include:

[0028] Obtain the workpiece region outline in each frame and obtain all corner points on the workpiece region outline; in each video frame sequence, take any frame as the target frame and any pixel in the target frame as the target corner point, find the nearest neighbor corner point of the target corner point in each non-target frame based on the KD tree, and use the target corner point and its nearest neighbor corner point in each non-target frame as a series of matching points.

[0029] Furthermore, the method for obtaining the corner points includes:

[0030] All corner points are obtained based on the Hessian corner detection algorithm.

[0031] The present invention also proposes a data transmission system for an industrial vision cloud service platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a data transmission method for an industrial vision cloud service platform.

[0032] The present invention has the following beneficial effects:

[0033] This invention first acquires video frame sequences of each workpiece from different inspection angles during the quality inspection process, and obtains the modeling and inspection time of each inspected workpiece to prepare for subsequent evaluation of the cloud platform's cloud performance parameters. Then, in each video frame sequence, based on the difference in matching pixels between adjacent frames, the relative change parameters of each frame are evaluated. Furthermore, by comprehensively considering the relative change parameters of frames with the same frame number in all video frame sequences during the quality inspection process of each workpiece, the frame blur coefficient under each frame number is evaluated. The frame blur coefficient will affect the subsequent adjustment of the compression rate to a certain extent. Furthermore, during the quality inspection process of each inspected workpiece, the cloud platform's cloud performance parameters are constructed to provide historical reference for subsequent analysis of the cloud platform's computing power. Then, during the quality inspection process of the current workpiece to be inspected, based on the frame blur coefficient under each frame number and the preset compression rate range, combined with the cloud performance parameters corresponding to the quality inspection processes of all inspected workpieces, the compression rate of each frame in each video frame sequence during the current quality inspection process is obtained. Finally, the corresponding frame is compressed and transmitted according to the compression rate. This invention analyzes the frame blurring of inspected workpieces during the quality inspection process and the modeling and quality inspection time to assess the historical computing power resources of the cloud platform, providing a reference for assessing the current computing power resources of the cloud platform. Then, it dynamically adjusts the frame compression rate based on the frame blurring of the workpiece to be inspected during the quality inspection process. This reduces transmission bandwidth while taking into account the working status of the cloud server in real time, improving the transmission effect of edge data and thus improving the subsequent quality inspection effect. Attached Figure Description

[0034] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart illustrating a data transmission method for an industrial vision cloud service platform according to an embodiment of the present invention;

[0036] Figure 2 This is a flowchart illustrating a method for obtaining compression ratio according to an embodiment of the present invention. Detailed Implementation

[0037] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a data transmission method and system for an industrial vision cloud service platform proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0038] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] The following description, in conjunction with the accompanying drawings, details the specific scheme of the data transmission method and system of the industrial vision cloud service platform provided by this invention.

[0040] Please see Figure 1 The diagram illustrates a data transmission method flowchart for an industrial vision cloud service platform according to an embodiment of the present invention, specifically including:

[0041] Step S1: Obtain video frame sequences of each workpiece from different inspection angles during the quality inspection process. The workpieces include the current workpiece to be inspected and the inspected workpieces. Obtain the modeling and inspection time of each inspected workpiece on the cloud platform.

[0042] To efficiently transmit the workpiece surface images and other quality inspection data collected by the automatic quality inspection equipment (hereinafter referred to as the inspection terminal) to the cloud platform for modeling and quality inspection, one embodiment of the present invention first collects workpiece surface videos from different inspection angles during the quality inspection process of each workpiece. The workpiece includes the current workpiece to be inspected and the workpiece that has already been inspected. Different inspection angles refer to different shooting angles of the workpiece. All inspection angles should cover the entire outer contour of the workpiece as much as possible to facilitate subsequent modeling and quality inspection.

[0043] It should be noted that one embodiment of the present invention mainly targets the surface quality inspection of engine turbocharger housing workpieces; in other embodiments, the implementer also performs surface quality inspection on other types of workpieces, and the compression and transmission method of quality inspection data and the surface quality inspection method are consistent for different workpieces.

[0044] It should be noted that collecting videos of workpieces from different shooting angles using existing quality inspection terminals is a well-known technology and will not be elaborated further.

[0045] In one embodiment of the present invention, after acquiring videos of each workpiece from different inspection angles during the quality inspection process, each video is divided into frames at a preset frame rate of 120fps to obtain all frame images in each video. Then, the frame images are sorted according to the acquisition time sequence to construct a video frame sequence corresponding to each video. The video frame sequence acquired by the quality inspection terminal and the parameters involved in subsequent analysis are used as edge data and then compressed and transmitted to the cloud platform.

[0046] It should be noted that the video length may vary from different quality inspection angles, and the corresponding video frame sequence length may also vary. For the convenience of subsequent analysis, the length of the video frame sequence is defined as the shortest length among all video frame sequences. Without affecting the modeling effect, frames exceeding the shortest length are discarded. Implementers may also define their own preset frame rate, but at least the frame rate of the video frame sequences from different quality inspection angles collected simultaneously must be the same.

[0047] In one embodiment of the present invention, after acquiring all video frame sequences of each inspected workpiece, these video frame sequences can be uploaded to a cloud server to perform 3D modeling of the inspected workpiece, providing an analytical and comparative basis for subsequent quality control, defect detection, surface roughness analysis, and other work; furthermore, the modeling and inspection time during the 3D modeling process is acquired to prepare for subsequent evaluation of the cloud platform's cloud performance parameters; thus enabling the quality inspection data transmission and quality inspection process to be adaptively adjusted according to the cloud platform's cloud performance parameters without affecting the quality inspection effect.

[0048] It should be noted that 3D modeling of workpieces based on video frame sequences from different angles is a well-known technology and will not be elaborated further.

[0049] Step S2: In each video frame sequence, different frame images are matched and tracked, and the relative change parameters of each frame image are obtained based on the difference of matching pixels between adjacent frame images; during the quality inspection of each workpiece, the frame image blur coefficient under each frame image number is obtained based on the relative change parameters of the frame images under the same frame image number in different video frame sequences.

[0050] Considering that when the robotic arm grasps and moves the workpiece, the video frames captured are prone to edge blurring due to the movement of the workpiece along with the robotic arm, and since the workpiece outline itself may be irregular in shape, the blurring of the frames will be further aggravated; the higher the degree of blurring of the frames, the less it contributes to subsequent modeling. Therefore, the compression rate can be appropriately increased to balance the transmission pressure, taking into account the cloud platform's cloud performance.

[0051] Furthermore, considering that the displacement of the robotic arm during the workpiece handling process is relatively limited, the changes between adjacent frames in each video frame sequence during the quality inspection of each workpiece should also be relatively limited. If each frame changes significantly relative to its adjacent frames, it indicates that there may be ambiguity.

[0052] Based on this, the embodiments of the present invention first match and track different frames in each video frame sequence, and then obtain the relative change parameters of each frame based on the difference of matching pixels between adjacent frames. The relative change parameters reflect the inter-frame changes of the workpiece during the flow process. Furthermore, the inter-frame changes in the video frame sequence under all quality inspection angles during the quality inspection process of each workpiece are comprehensively considered, that is, the relative change parameters of frames with the same frame number are comprehensively considered to evaluate the frame blur coefficient under each frame number during the quality inspection process. The frame blur coefficient reflects the motion blur at the acquisition time corresponding to each frame number, which prepares for adjusting the compression and transmission method in combination with the cloud performance of the cloud platform.

[0053] Preferably, in one embodiment of the present invention, considering that there may be many corner points in the workpiece, direct matching and tracking would consume a large amount of computing resources. Furthermore, considering that the contour edges are most prone to blurring, comparing the relative changes of corner points on the contour edges can also provide some blurring reference value. Therefore, the corner points on the contour edges of the workpiece in each frame are first obtained, and then matched and tracked to evaluate the changes between frames. Also, considering that KD-trees (K-Dimensional Trees) can efficiently process high-dimensional data and perform fast searches in multi-dimensional space, they can improve the matching speed of corner points between different frames. Based on this, the method for matching and tracking different frames includes:

[0054] Obtain the workpiece region outline in each frame and all corner points on the workpiece region outline; in each video frame sequence, take any frame as the target frame and any pixel in the target frame as the target corner point, find the nearest neighbor corner point of the target corner point in each non-target frame based on the KD tree, and use the target corner point and its nearest neighbor corner point in each non-target frame as a series of matching points.

[0055] As an example, the workpiece region contour is first annotated based on a pre-trained contour annotation model, and all corner points on the workpiece region contour are obtained using the Hessian corner detection algorithm, along with a descriptor for each corner point. Then, a KD tree is constructed based on the descriptors of all corner points in each non-target frame. The nearest neighbor corner point of the target corner point in the target frame is queried in the KD tree corresponding to each non-target frame, thus obtaining the matching corner point of the target corner point in each non-target frame. The nearest neighbor distance ratio of the KD tree is set to 0.75, but the implementer can also define it themselves.

[0056] It should be noted that the training and application of the contour annotation model, the Hessian corner detection and descriptor acquisition, and the feature matching based on KD trees are all existing technologies and will not be elaborated here. In other examples, implementers can also obtain all edge contours in each frame using edge detection algorithms such as the Canny algorithm, and further use the SIFT algorithm to obtain the corners and corresponding descriptors on the edge contours. Implementers can also adjust the feature matching algorithm based on the computing power of the quality inspection terminal server, i.e., the edge server, such as directly using the SIFT algorithm for feature matching, or using optical flow to track and match pixels. These are all well-known technologies and will not be elaborated here.

[0057] After obtaining the matching points between different frames in each video frame sequence, the relative change parameters of each frame can be calculated.

[0058] Preferably, in one embodiment of the present invention, considering that the displacement of the robotic arm is relatively limited and the frame rate is relatively high, the positional difference of the matching corner points between adjacent frames in each video frame sequence reflects the changes to a certain extent, and indirectly reflects the blurring effect; therefore, the method for obtaining the relative change parameter includes:

[0059] In each video frame sequence, a Cartesian coordinate system is constructed with the center of each frame as the origin to obtain the position coordinates of each pixel in each frame. Between each frame and the previous adjacent frame, the Euclidean distance between the position coordinates of each pair of matching pixels is used as a variation sub-parameter, and the average of the variation sub-parameters between all matching pixels is used as the relative variation parameter between each frame and the previous adjacent frame.

[0060] In another embodiment of the present invention, between each frame and the previous adjacent frame, the absolute value of the difference between the gray values ​​of each pair of matched pixels is further calculated, and the fusion result of the absolute value of the gray value difference and the Euclidean distance, such as the product, sum or weighted sum, is further used as the change sub-parameter to obtain the relative change parameter.

[0061] After obtaining the relative change parameters of each frame in each video frame sequence, the frame blur coefficient of each workpiece under each frame number in the quality inspection process can be further obtained.

[0062] Preferably, in one embodiment of the present invention, the method for obtaining the frame blur coefficient includes:

[0063] During the quality inspection process of each workpiece, between every two video frame sequences, the absolute value of the difference between the relative change parameters between frames with the same frame number is used as the fuzzy sub-parameter for the corresponding frame number.

[0064] Under each frame number, the fuzziness coefficient is obtained by combining the fuzzy sub-parameters between all two different video frame sequences during the workpiece quality inspection process.

[0065] As an example, during the quality inspection of each workpiece, the blur sub-parameters under each frame number between every two video frame sequences are first obtained; then, the blur sub-parameters under each frame number between all the pairwise video frame sequences are accumulated, and the linearly normalized sum of the accumulated sum is used as the frame blur coefficient under the corresponding frame number; the greater the difference between the pairwise combinations, the greater the difference in frame changes at the same acquisition time under any two quality inspection angles, and the more likely there is edge blurring.

[0066] In another embodiment of the present invention, the implementer may also calculate the variance of the relative change parameters of the frames with the same frame number in all video frame sequences during the quality inspection process of each workpiece, normalize the variance, and obtain the frame blur coefficient under the corresponding frame number; the larger the normalized value of the variance, the greater the fluctuation of the frame change at the same acquisition time under any different quality inspection angle, the greater the consistency, and the more likely there is edge blur.

[0067] It should be noted that implementers may also use other normalization methods, or use other discreteness metrics such as standard deviation to replace variance. These are all existing technologies and will not be elaborated further.

[0068] Step S3: During the quality inspection process of each inspected workpiece, construct the cloud performance parameters of the cloud platform based on the frame blur coefficient and modeling detection time under all frame sequence numbers; During the quality inspection process of the current workpiece to be inspected, obtain the compression rate of each frame in each video frame sequence during the current quality inspection process based on the frame blur coefficient and preset compression rate range under each frame sequence number, combined with the cloud performance parameters corresponding to the quality inspection process of all inspected workpieces.

[0069] Considering that shorter modeling and inspection times in each quality inspection process indicate stronger computing power of the cloud platform, the compression rate of subsequent workpiece video frames can be appropriately reduced to improve modeling and quality inspection accuracy; conversely, longer modeling and inspection times indicate weaker computing power of the cloud platform, and the compression rate of workpiece video frames can be appropriately increased to improve data transmission efficiency and the efficiency between subsequent decompression processing and modeling. Furthermore, considering that frames with larger blur coefficients contribute less to subsequent modeling in each quality inspection process, the compression rate can be appropriately increased to improve transmission efficiency; however, increasing the compression rate will lead to subsequent decompression consuming significant cloud platform computing resources. Therefore, it is necessary to comprehensively evaluate the cloud platform's cloud performance parameters during the quality inspection process of each inspected workpiece, thereby preparing for determining the compression rate of the frames.

[0070] Preferably, in one embodiment of the present invention, considering that during the quality inspection process of each inspected workpiece, the larger the frame fuzziness coefficient under each frame number, the higher the frame compression may be to balance the transmission pressure during this quality inspection, which may also put greater pressure on the cloud computing platform, resulting in smaller cloud performance parameters; the longer the modeling and inspection time, the greater the computing power pressure on the cloud platform during this quality inspection process, which is insufficient for rapid modeling, and the smaller the cloud performance parameters; therefore, the method for obtaining cloud performance parameters includes:

[0071] During the quality inspection of each inspected workpiece, the negative correlation mapping result of the sum of the frame fuzziness coefficients under all frame sequence numbers is used as the first performance parameter, and the negative correlation mapping result of the modeling and inspection time is used as the second performance parameter. The first performance parameter and the second performance parameter are used as vector elements to construct a two-dimensional vector, and the two-dimensional vector is used as the cloud performance parameter of the cloud platform during the quality inspection process of the corresponding known workpiece.

[0072] As an example, in the quality inspection process of each inspected workpiece, the sum of the frame blur coefficients under all frame sequence numbers is counted in reverse to obtain the first performance parameter. The larger the frame blur coefficient, the higher the compression rate should be, and the greater the cloud computing pressure on the subsequent cloud platform. Therefore, the smaller the first performance parameter is, the smaller the first performance parameter is. Then, the modeling and inspection time is also counted in reverse to obtain the second performance parameter. The longer the modeling and inspection time, the weaker the current cloud computing capability of the cloud platform. Therefore, the second performance parameter is also smaller.

[0073] Considering the cloud performance parameters corresponding to the quality inspection process of all inspected workpieces, it can provide a certain historical reference for the cloud performance of the cloud platform during the quality inspection process of the current workpiece to be inspected. This helps to evaluate the degree of adjustment of the compression rate by combining the frame blur coefficients of each frame number in the quality inspection process of the current workpiece to be inspected. Finally, it combines the preset compression rate range to determine the compression rate of each frame in each video frame sequence during the current quality inspection process.

[0074] Preferably, in one embodiment of the present invention, the method for obtaining the compression ratio includes:

[0075] Please see Figure 2 The diagram illustrates a flowchart of a method for obtaining compression ratio according to an embodiment of the present invention, specifically including:

[0076] Step S201: Integrate the first performance parameter and the second performance parameter in the cloud performance parameters of the cloud platform during the quality inspection process of each inspected workpiece to obtain the computational redundancy parameter of the cloud platform in each quality inspection process; Based on the computational redundancy parameter in the quality inspection process of all inspected workpieces, obtain the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected.

[0077] Considering that the larger the cloud performance parameter, the stronger the computing redundancy capability of the cloud platform in the corresponding quality inspection process, the first performance parameter and the second performance parameter are first fused to obtain the corresponding computing redundancy parameter. As an example, the first performance parameter and the second performance parameter are added and fused. In other examples, implementers can also use basic mathematical operations or related mapping methods such as multiplication or weighted fusion to fuse the two, which will not be elaborated further.

[0078] Furthermore, considering the computational redundancy parameters of the cloud platform between the previous and next adjacent processes of the workpiece to be inspected, which can reflect the computing power of the cloud platform in the short term, the larger the computational redundancy parameter of the previous process, the larger the computational redundancy parameter of the cloud platform may be in the current quality inspection process. Also considering the fluctuation characteristics of the computational redundancy parameters corresponding to all quality inspection processes, which indirectly reflects the performance stability of the cloud platform, the smaller the difference in the changes of computational redundancy parameters in historical quality inspection processes, the more stable the performance of the cloud platform. When the computational redundancy parameter of the cloud platform is larger in the current quality inspection process, the computational redundancy stability coefficient is also larger, which indicates that the performance of the cloud platform is more likely to be stable in the current quality inspection process.

[0079] Based on this, in a preferred embodiment of the present invention, the method for obtaining the cloud computing capacity coefficient includes:

[0080] The computational redundancy parameters of the cloud platform in the previous adjacent quality inspection process of the current workpiece to be inspected are used as the current computational redundancy parameters; based on the fluctuation deviation of the computational redundancy parameters in the quality inspection process of each inspected workpiece relative to the average level, the computational redundancy stable parameters are obtained.

[0081] By integrating the current computational redundancy parameters and the computational redundancy stability parameters, the cloud computing capacity coefficient of the current workpiece to be inspected during the quality inspection process is obtained.

[0082] As an example, the formula for calculating the cloud computing capacity factor is:

[0083] Where R is the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected; j is the sequence number of the quality inspection process of the inspected workpiece; n j This is the sequence number of the previous adjacent quality inspection process for the workpiece to be inspected, and also the total number of quality inspection processes for the workpieces that have already been inspected. This refers to the computational redundancy parameters of the cloud platform in the previous adjacent quality inspection process of the workpiece to be inspected, and also the current computational redundancy parameters; D j Let μ be the computational redundancy parameter of the cloud platform during the quality inspection process of the j-th inspected workpiece; μ is the mean of the computational redundancy parameters of the cloud platform during the quality inspection process of all inspected workpieces; || is the absolute value sign; exp() is the exponential function with the natural constant e as the base. These are the computational redundancy and stability parameters for the cloud platform.

[0084] In the above formula, the fluctuation deviation of each computational redundancy parameter relative to the mean is evaluated by calculating the absolute value of the difference between each parameter and the mean. The larger the sum of the absolute values ​​of the differences, the more volatile the computational redundancy parameter is. Therefore, the negative correlation is mapped to the exponential function to adjust the logic and normalize it, so that the computational redundancy stable parameter is smaller. Then, the computational redundancy stable parameter is multiplied and combined with the current computational redundancy parameter to obtain the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected.

[0085] In other examples, implementers can also assess volatility based on the variance or standard deviation of all computationally redundant parameters, and then perform negative correlation normalization to obtain computationally redundant stable parameters; implementers can also use other negative correlation normalization methods, such as taking the reciprocal, which will not be elaborated further.

[0086] Step S202: Combine the negative correlation mapping results of the cloud computing capacity coefficient in the quality inspection process of the current workpiece under inspection with the frame image blur coefficient under each frame image number to obtain the frame image compression weight under each frame image number in all video frame sequences of the current workpiece under inspection.

[0087] Considering that if the cloud computing capacity coefficient of the current workpiece to be inspected is larger, the compression ratio can be appropriately increased to improve transmission efficiency. At this time, the cloud computing capacity is sufficient for subsequent decompression and modeling. Also considering that if the frame blur coefficient of the frame in the current workpiece to be inspected is higher, it means that its contribution to subsequent modeling is lower, so the compression ratio can also be appropriately increased. Based on this, the frame compression weight under each frame number in all video frame sequences of the current workpiece to be inspected can be obtained.

[0088] As an example, the formula for calculating frame compression weights is:

[0089] Where m is the frame number in the sequence of all video frames of the workpiece to be inspected; P represents the frame compression weight for the m-th frame in the sequence of all video frames of the workpiece under inspection. m R is the frame blur coefficient of the m-th frame in the sequence of all video frames of the workpiece to be inspected; R is the cloud computing capacity coefficient in the quality inspection process of the workpiece to be inspected.

[0090] In the above formula, the capacity coefficient is calculated by adding 1 and then performing a reciprocal operation. The reciprocal is then multiplied and fused with the frame image blur coefficient to obtain the frame image compression weight. The frame image compression weight reflects the adjustment range of the compression rate of the frame image in the future. The larger the weight, the more the compression rate should be adjusted, and vice versa. In other examples, other negative correlation mapping methods can also be used, which will not be elaborated here.

[0091] It should be noted that since the frame blur coefficient obtained in the aforementioned steps has a value range of 0-1, the frame compression weight also has a value range of 0-1, in preparation for obtaining the compression ratio later.

[0092] Step S203: Based on the frame compression weight and the preset compression rate range, obtain the compression rate of each frame in each video frame sequence during the current quality inspection process.

[0093] In a preferred embodiment of the present invention, the method for obtaining the compression ratio of each frame includes:

[0094] The left endpoint of the preset compression rate range is used as the base compression rate. The length of the preset compression rate range is weighted using frame compression weights, and the weighted result is used as the adjustment compression rate. The base compression rate is added to the adjustment compression rate to obtain the compression rate of the frame at the corresponding frame number.

[0095] As an example, the formula for calculating compression ratio is:

[0096] Where m is the frame number in the sequence of all video frames of the workpiece to be inspected; L m The compression ratio of the frame at the m-th frame number in the video frame sequence of the current workpiece to be inspected; is the frame compression weight of the m-th frame in the sequence of all video frames of the workpiece to be inspected; a is the left endpoint of the preset compression rate range, which is also the base compression rate; b is the right endpoint of the preset compression rate range; ba is the length of the preset compression rate range. Adjust the compression rate of the frame at the m-th frame number in the video frame sequence of the current workpiece to be inspected.

[0097] It should be noted that the preset compression rate range needs to be determined in conjunction with the accuracy requirements of the frame graph during subsequent modeling. In this example, it is set to 80%-90%, but implementers can also define it themselves.

[0098] Step S4: Compress the corresponding frame image according to the compression ratio and transmit the compression result to the cloud platform.

[0099] After obtaining the frame image under each frame number, the corresponding frame image can be compressed to obtain a compressed image. Then, the compressed images are sorted according to the frame image requirements to obtain all compressed video frame sequences. The compressed video frame sequence and the frames in the video frame sequence are then compared. Figure 1One-to-one correspondence; then, based on 5G communication technology and TCP / IP protocol, all compressed video frame sequences of the current workpiece to be inspected are transmitted to the cloud server, then decompressed, and then 3D modeled for the current workpiece to be inspected. The surface quality of the current workpiece to be inspected is evaluated based on the 3D model. It should be noted that 5G communication transmission and 3D modeling are both well-known technologies and will not be elaborated further.

[0100] The present invention also proposes a data transmission system for an industrial vision cloud service platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a data transmission method for an industrial vision cloud service platform.

[0101] In summary, this invention acquires video frame sequences of each workpiece from different inspection angles during the quality inspection process, and obtains the modeling and inspection time of each inspected workpiece on the cloud platform. Then, during the quality inspection of each workpiece, it analyzes and evaluates the relative change parameters of each frame in each video frame sequence, further obtaining the frame blur coefficient under each frame number, and constructs the cloud platform's cloud performance parameters based on the modeling and inspection time. Then, based on the frame blur coefficient under each frame number and a preset compression rate range, combined with the cloud performance parameters corresponding to the quality inspection process of all inspected workpieces, it obtains the compression rate of each frame in each video frame sequence during the current quality inspection process, and then compresses and transmits the corresponding frame. This invention analyzes the frame blur situation and modeling and quality inspection time of inspected workpieces during the quality inspection process, assesses the historical computing power resources of the cloud platform, and provides a reference for assessing the current computing power resources of the cloud platform. Then, it dynamically adjusts the frame compression rate based on the frame blur situation of the current workpiece to be inspected during the quality inspection process, reducing transmission bandwidth while taking into account the real-time working conditions of the cloud server, improving the transmission effect of edge data, and thus improving the subsequent quality inspection effect.

[0102] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A data transmission method of an industrial vision cloud service platform, characterized in that, The method comprises: acquiring video frame sequences of each workpiece at different quality inspection angles in the quality inspection process, the workpiece including a current workpiece to be inspected and an inspected workpiece; and acquiring a modeling detection duration of each inspected workpiece on a cloud platform; in each video frame sequence, matching and tracking different frame graphs, and acquiring a relative change parameter of each frame graph according to the difference between matching pixel points between adjacent frame graphs; in the quality inspection process of each workpiece, acquiring a frame graph blur coefficient of each frame graph sequence number according to the relative change parameter of the frame graph under the same frame graph sequence number in different video frame sequences; in the quality inspection process of each inspected workpiece, constructing a cloud performance parameter of the cloud platform according to the frame graph blur coefficient under all frame graph sequence numbers and the modeling detection duration; in the quality inspection process of the current workpiece to be inspected, acquiring a compression rate of each frame graph in each video frame sequence in the current quality inspection process according to the frame graph blur coefficient under each frame graph sequence number and a preset compression rate range, and combining the cloud performance parameter corresponding to the quality inspection process of all inspected workpieces; compressing the corresponding frame graph according to the compression rate, and transmitting the compression result to the cloud platform. 2.The data transmission method of the industrial visual cloud service platform of claim 1, wherein, The method for acquiring the relative change parameter comprises: in each video frame sequence, constructing a rectangular coordinate system with the center of each frame graph as the origin, and acquiring the position coordinates of each pixel point in each frame graph; between each frame graph and the previous adjacent frame graph, taking the Euclidean distance between the position coordinates of each pair of matching pixel points as a change sub-parameter, and taking the mean value of the change sub-parameters between all matching pixel points as the relative change parameter between each frame graph and the previous adjacent frame graph. 3.The data transmission method of the industrial visual cloud service platform of claim 1, wherein, The method for acquiring the frame graph blur coefficient comprises: in the quality inspection process of each workpiece, between each two video frame sequences, taking the absolute value of the difference between the relative change parameters of the frame graphs under the same frame graph sequence number as a blur sub-parameter under the corresponding frame graph sequence number; under each frame graph sequence number, synthesizing the blur sub-parameters between all two different video frame sequences in the quality inspection process of the workpiece to acquire the frame graph blur coefficient.

4. The data transmission method of the industrial visual cloud service platform according to claim 1, characterized in that, The method for acquiring the cloud performance parameter comprises: in the quality inspection process of each inspected workpiece, taking the negative correlation mapping result of the sum value of the frame graph blur coefficients under all frame graph sequence numbers as a first performance parameter, and taking the negative correlation mapping result of the modeling detection duration as a second performance parameter; taking the first performance parameter and the second performance parameter as vector elements to construct a two-dimensional vector, and taking the two-dimensional vector as the cloud performance parameter of the cloud platform in the quality inspection process of the corresponding known workpiece.

5. The data transmission method of claim 4, wherein, The method for acquiring the compression rate comprises: fusing the first performance parameter and the second performance parameter in the cloud performance parameter of the cloud platform in the quality inspection process of each inspected workpiece to obtain a calculation redundancy parameter of the cloud platform in each quality inspection process; and acquiring a cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected according to the calculation redundancy parameter in the quality inspection process of all inspected workpieces. Fuse the negative correlation mapping result of the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected with the frame graph blur coefficient under each frame graph sequence number to obtain the frame graph compression weight under each frame graph sequence number in all video frame sequences of the current workpiece to be inspected; According to the frame graph compression weight and the preset compression rate range, the compression rate of each frame graph in each video frame sequence in the current quality inspection process is obtained.

6. The data transmission method of claim 5, wherein, The method for obtaining the cloud computing capacity coefficient comprises: The computing redundancy parameter of the cloud platform in the last adjacent quality inspection process of the current workpiece to be inspected is taken as the current computing redundancy parameter; the computing redundancy stability parameter is obtained according to the fluctuation deviation of the computing redundancy parameter in the quality inspection process of each inspected workpiece relative to the average level; Fuse the current computing redundancy parameter and the computing redundancy stability parameter to obtain the cloud computing capacity coefficient in the quality inspection process of the current workpiece to be inspected.

7. The data transmission method of claim 5, wherein, The method for obtaining the compression rate of each frame graph in each video frame sequence in the current quality inspection process according to the frame graph compression weight and the preset compression rate range comprises: The left end point of the interval of the preset compression rate range is taken as the basic compression rate, the interval length of the preset compression rate range is weighted by using the frame graph compression weight, and the weighted result is taken as the adjustment compression rate; the basic compression rate is added to the adjustment compression rate to obtain the compression rate of the frame graph under the corresponding frame graph sequence number. 8.The data transmission method of the industrial visual cloud service platform of claim 1, wherein, The method for matching and tracking different frame graphs comprises: Obtain the workpiece region contour in each frame graph and all corner points on the workpiece region contour; in each video frame sequence, any frame graph is taken as a target frame graph, any pixel point in the target frame graph is taken as a target corner point, the nearest neighbor corner point of the target corner point in each non-target frame graph is found based on the KD tree, and the target corner point and the nearest neighbor corner point of the target corner point in each non-target frame graph are taken as a series of matching points. 9.The data transmission method of the industrial visual cloud service platform of claim 8, wherein, The method for obtaining the corner point comprises: All corner points are obtained based on the Hessian corner detection algorithm.

10. A data transmission system of an industrial vision cloud service platform, characterized by, The computer program stored in the memory and executable on the processor comprises the steps of the data transmission method of the industrial visual cloud service platform according to any one of claims 1-9.

Images