Industrial quality inspection data cooperative transmission method driven by image recognition

Abstract

The invention relates to an industrial quality inspection data cooperative transmission method driven by image recognition, in particular to the field of artificial intelligence, which analyzes video frames in real time through a lightweight neural network, accurately locates and extracts key areas and features in industrial product images, constructs a dynamic semantic scene graph to understand contents and divide priorities, and improves the quality inspection efficiency. The core of the method is that non-uniform intelligent coding is carried out on video streams according to semantic importance of contents, high-fidelity transmission of key information such as defects is ensured, non-key areas are greatly compressed to save bandwidth, scheduling is carried out in the transmission process according to priorities, confidence feedback based on a cloud recognition result is introduced, closed-loop control is formed, and high-fidelity transmission is realized. According to the method, the front-end analysis model, the coding strategy and the network path are adaptively optimized, so that low-delay and high-reliability visual quality inspection data transmission and accurate identification can be continuously and stably realized in a complex industrial network environment, and the overall efficiency and the intelligent level of online quality inspection are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to an image recognition-driven collaborative transmission method for industrial quality inspection data. Background Technology

[0002] With the continuous improvement of intelligent manufacturing in industry, online quality inspection technology based on machine vision is playing an increasingly crucial role in high-end manufacturing fields such as semiconductor wafers and precision components. In such high-speed production environments, multiple high-frame-rate, high-resolution industrial cameras are typically deployed to continuously acquire product surface images at a rate of hundreds of frames per second or even higher, forming a continuous high-definition or even ultra-high-definition video stream. The production line is extremely fast-paced, processing hundreds of products per minute. This requires the vision system to complete the transmission, analysis, and feedback of images in a very short time. At the same time, the network environment on the production site is complex and variable. Although advanced communication technologies such as Wi-Fi 6 or 5G private networks are adopted, the actual available bandwidth often fluctuates significantly due to the concurrent data traffic of other production systems (such as AGV scheduling, equipment monitoring, and environmental perception) and the influence of physical factors such as metal structures and electromagnetic interference in the factory. This scenario of high data throughput demand and unstable network conditions poses an extremely severe challenge to the real-time, reliable, and low-latency transmission of video data.

[0003] To address the aforementioned transmission pressures, existing technologies generally employ standard video coding and compression schemes, such as H.264 or H.265 encoding based on fixed or variable bitrates, to initially compress the video stream before transmission. Alternatively, in extreme cases, video frames may be periodically thinned to reduce data volume. However, these methods have fundamental limitations. Fixed bitrate strategies struggle to adapt to dynamic changes in network conditions, easily leading to packet loss or increased transmission delays during network congestion. While variable bitrate strategies offer some improvement, their bitrate allocation is not correlated with the value of the image content itself. More importantly, the core objective of industrial quality inspection is to accurately identify surface defects in products, but traditional encoders are designed based on human visual perception. Guided by optimization, its bitrate allocation does not consider the specific needs of defect detection algorithms. As a result, a large amount of bandwidth may be consumed in irrelevant background or uniform areas in the image, while the real critical areas—such as tiny scratches, foreign objects, or deformations—may lose subtle features due to compression distortion, directly affecting the accuracy of the recognition model. In addition, simple frame thinning operations have a huge risk of missing transient defects. Therefore, the core problem of the current technology system is the lack of a mechanism that can understand the importance of video content to the recognition task in real time and dynamically adjust the encoding parameters and transmission strategies based on this importance, so as to achieve an intelligent transmission scheme that coordinates content awareness, network adaptation, and recognition accuracy assurance. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing an image recognition-driven collaborative transmission method for industrial quality inspection data, thereby resolving the issues raised in the background section.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: specifically, it includes the following steps: Step S1: Analyze the acquired video frames in real time using a lightweight neural network deployed on an industrial camera or edge computing node to generate a first visual importance heatmap representing the salience of key regions, and extract the corresponding first intrinsic feature descriptors for the key regions. Step S2: Based on the first visual importance heatmap and the first intrinsic feature descriptor, construct a dynamically updated semantically enhanced MPEG-4 scene graph. The semantically enhanced MPEG-4 scene graph includes node attributes, spatiotemporal relationships, and outputs first partition metadata containing region type identifier and transmission priority. Step S3: Based on the first partition metadata output by the semantically enhanced MPEG-4 scene graph, and based on the preset bitrate mapping rules, perform non-uniform bitrate distribution encoding on the video frames. The key areas identified by the first partition metadata are encoded with low compression ratio, while the non-key areas are encoded with high compression ratio or skipped encoding. A collaborative data packet containing the encoded video basic stream and the first partition metadata is generated. Step S4: The collaborative controller schedules the data transmission of the collaborative data packet according to the transmission priority defined by the first partition metadata in the collaborative data packet, and receives the confidence feedback returned by the cloud after identifying the transmission data. Based on the confidence feedback, the network parameters of the lightweight neural network in step S1, the bit rate mapping rules in step S3, or the transmission path decision of the collaborative data packet are dynamically adjusted to form a closed-loop optimization. In a preferred embodiment, in step S1, the real-time analysis of video frames by the lightweight neural network includes multi-scale feature fusion and spatial attention weighting, specifically: The lightweight neural network employs a cascaded dilated convolutional pyramid structure. By setting dilated convolutional layers with different dilation rates in parallel, it simultaneously extracts depth feature maps of the input video frame under three different receptive fields, denoted as the first-scale feature map, the second-scale feature map, and the third-scale feature map, respectively. Subsequently, the first-scale feature map, the second-scale feature map, and the third-scale feature map are fused through a channel-space dual attention unit. The channel attention mechanism first performs global average pooling on each first-scale feature map, second-scale feature map, or third-scale feature map, and generates an importance weight vector for the feature channel through a fully connected layer for recalibrating the feature channels. The spatial attention mechanism, based on the spatial context relationship of the first-scale feature map, second-scale feature map, and third-scale feature map, generates a spatial weight matrix with the same resolution as the first-scale feature map, second-scale feature map, or third-scale feature map through a convolutional layer. Next, the first-scale feature map, the second-scale feature map, and the third-scale feature map, which have been recalibrated by the channel importance weight vector, are multiplied element-wise with the spatial weight matrix to achieve weighted fusion and obtain a unified feature map. Based on a unified feature map, a convolutional layer with a kernel size of 1x1 is used for processing, and a sigmoid activation function is applied to map each pixel value of the processing result to a closed interval between zero and one, thereby generating a single-channel first visual importance heatmap with the same spatial resolution as the input video frame. The value of each pixel in the first visual importance heatmap represents the visual saliency probability of its belonging to a key region, and this value is the saliency probability value of the pixel. The channel importance weight vector, spatial weight matrix, and parameters of the convolutional layer together constitute the network parameters of the lightweight neural network.

[0006] In a preferred embodiment, the specific process of extracting the corresponding first intrinsic feature descriptor for the key region is as follows: First, an adaptive threshold segmentation algorithm is applied to the first visual importance heatmap. The segmentation threshold is dynamically calculated based on the overall distribution of the first visual importance heatmap. Pixels with a saliency probability value higher than the threshold in the first visual importance heatmap are classified as key pixels, forming a binary mask. In the binary mask, all connected key regions are identified through connected component analysis. Each key region represents a continuous potential defect or main part of the product. Then, for each key region, within the corresponding spatial range of the unified feature map generated by the lightweight neural network in step S1, a region feature aggregation operation is performed to calculate the first intrinsic feature descriptor of the key region. Its value is: multiplying the feature vector corresponding to each pixel in the unified feature map within the key region by the saliency probability value of the pixel in the first visual importance heatmap, and summing all the product feature vectors to form a sum vector; then dividing this sum vector by a normalization factor, which is the sum of the saliency probability values ​​of all pixels in the key region in the first visual importance heatmap, plus a positive constant.

[0007] In a preferred embodiment, the specific process of constructing the dynamically updated semantically enhanced MPEG-4 scene graph in step S2 is as follows: First, based on the first visual importance heatmap and the first intrinsic feature descriptor generated in step S1, each key region is subjected to node attribute processing, where node attributes include spatial attributes, semantic attributes, and dynamic weights. The spatial attributes are determined as follows: the abscissa of each pixel in the key region is calculated by multiplying it by the saliency probability value of that pixel in the first visual importance heatmap, and all products are summed to obtain a weighted sum of abscissas; the ordinate of each pixel in the key region is calculated by multiplying it by the saliency probability value of that pixel in the first visual importance heatmap, and all products are summed to obtain a weighted sum of ordinates; the weighted sum of abscissas and the weighted sum of ordinates are divided by the sum of the saliency probability values ​​of all pixels in the key region, and the two quotients are used as the abscissa and ordinate of the center position of the region, respectively; the semantic attributes are obtained by classifying the first intrinsic feature descriptor using a lightweight classifier and are used to identify the region type; the value of the dynamic weight is the largest saliency probability value in the first visual importance heatmap among all pixels contained in the key region. Secondly, semantic layer nodes and virtual coordination nodes are added to the standard MPEG-4 scene graph structure; the semantic layer nodes are used to characterize the processing procedure status or defect severity level; the virtual coordination nodes are used to aggregate the shared features of multiple physical key area nodes in the same processing procedure. A spatiotemporal correlation is established by constructing cross-frame edges that represent the spatiotemporal correlation through the calculation of feature similarity between adjacent frame nodes. The feature similarity is calculated as follows: first, a first Mahalanobis distance component is calculated, which is based on the difference between the first intrinsic feature descriptors corresponding to adjacent frame nodes and the inverse of a feature covariance matrix; then, a second cosine similarity component is calculated, which is the dot product of the two first intrinsic feature descriptors divided by the product of their respective moduli; finally, the result of transforming the first Mahalanobis distance component through a negative exponential function is multiplied by the second cosine similarity component to obtain the final feature similarity. If the feature similarity exceeds a preset threshold, a bidirectional edge is established. Finally, the semantically enhanced MPEG-4 scene graph is dynamically updated using an incremental graph convolutional network to optimize the relationship between node attributes and the edges represented by the cross-frame edges. The dynamic update mechanism is as follows: when a new frame is added to the acquired video frame, for the nodes in the semantically enhanced MPEG-4 scene graph that need to be updated, their updated attribute representations are calculated. The calculation process for this attribute representation is as follows: for each neighbor node of the node, the attribute representation of that neighbor node at the current time is taken, multiplied by a trainable weight matrix, and then multiplied by a normalization coefficient, which is the product of the square root of the size of the neighbor set of the node and its neighbor nodes. Then, the results of such multiplications of all neighbor nodes are summed, and finally a non-linear activation function is applied to the summation result. This update process is only performed on newly added nodes, nodes whose attributes have changed, and their neighbor nodes in the semantically enhanced MPEG-4 scene graph. The updated semantically enhanced MPEG-4 scene graph output includes first partition metadata containing region type identifiers and transmission priorities, where the transmission priority is calculated by weighting the dynamic weights and the temporal stability factor.

[0008] In a preferred embodiment, the specific process of outputting the first partition metadata, which includes the region type identifier and transmission priority, is as follows: Based on the node attributes of the semantically enhanced MPEG-4 scene graph, the region type identifier is classified into critical defect region, product main body region or background region; the classification is based on the output of the lightweight classifier in the node semantic attributes and is verified by combining the relative position of the key region in the spatial attributes. The transmission priority is calculated using a linear weighted formula. The specific calculation process is as follows: A weighted sum is calculated, where the first term of the sum is the dynamic weight multiplied by a first weight coefficient, and the second term is the temporal stability factor multiplied by a second weight coefficient. The temporal stability factor is calculated as follows: For the same node that has established a tracking relationship through the bidirectional edge, the Euclidean distance between the node's first intrinsic feature descriptor in the current frame and the corresponding first intrinsic feature descriptors in previous consecutive frames is calculated, resulting in multiple distance values. The arithmetic mean of these distance values ​​is then calculated. Finally, this arithmetic mean is subtracted from the calculated value, and the result is the temporal stability factor. The metadata of the first partition is stored in a structured form, including a unique identifier for the region, a region type identifier, a transmission priority, and spatial coordinate information.

[0009] In a preferred embodiment, in step S3, the video frames are encoded using non-uniform bitrate allocation based on a preset bitrate mapping rule. The bitrate mapping rule consists of three sub-rules: basic offset allocation, nonlinear modulation, and global constraints. The specific process is as follows: First, execute the basic offset allocation sub-rule: Based on the region type identifier in the first partition metadata, allocate a basic quantization parameter offset to each critical region; specifically: when the region type identifier is a critical defect region, the basic quantization parameter offset allocated to this region is a first negative integer; when the region type identifier is a product main body region, the basic quantization parameter offset allocated to this region is a second negative integer, the absolute value of which is less than the absolute value of the first negative integer; when the region type identifier is a background region, the basic quantization parameter offset allocated to this region is a positive integer. Secondly, execute the nonlinear modulation sub-rule: combine the transmission priority in the metadata of the first partition with the nonlinear modulation of the basic quantization parameter offset; Finally, the global constraint sub-rules are executed: based on the network bandwidth budget, the overall bitrate of the current video frame to be encoded is constrained for compliance; after generating the quantization parameter mapping table using the basic offset allocation sub-rules and nonlinear modulation sub-rules, the overall bitrate to be generated by encoding according to the quantization parameter mapping table is predicted; if the predicted overall bitrate exceeds the target threshold, the gradient redistribution strategy is initiated, the specific process of which is as follows: a global quantization parameter adjustment amount is introduced, and a local adjustment amount affected by the priority is calculated for each key region based on the transmission priority in the first partition metadata; the local adjustment amount is equal to the global quantization parameter adjustment amount, multiplied by an attenuation coefficient, and then multiplied by the difference obtained by subtracting the transmission priority of the region; the final quantization parameter value used for encoding in each key region is obtained by adding the global basic quantization parameter value configured by the video encoder, the final quantization parameter offset obtained by the nonlinear modulation sub-rules for that region, and the local adjustment amount; the constrained quantization parameter mapping table is sent to the encoder to drive the partition differential encoding of the video frame.

[0010] In a preferred embodiment, the specific operation of the nonlinear modulation sub-rule is as follows: The final quantization parameter offset for each critical region is calculated as follows: the base quantization parameter offset for that region plus an adjustment term. The adjustment term is calculated as follows: First, an intermediate value is calculated, which is equal to the transmission priority of that region minus 0.5. Then, a negative exponential function with the natural constant as the base is applied to this intermediate value, the result is added by 1, the reciprocal is taken, and then 0.5 is subtracted to obtain a modulation factor. After that, this modulation factor is multiplied by a preset modulation intensity coefficient, and then multiplied by the absolute value of the base quantization parameter offset for that region. The result is the adjustment term.

[0011] In a preferred embodiment, the process of generating a cooperative data packet containing the encoded video base stream and the metadata of the first partition specifically includes: The video encoder compresses video frames according to the quantization parameter mapping table used to drive the differential coding of video frame partitions and outputs the video basic stream; the video basic stream is encapsulated with the first partition metadata to generate a cooperative data packet; The multiplexed structure of the cooperative data packet includes a synchronization header and a timestamp. The first partition metadata in the cooperative data packet embeds a checksum to detect transmission errors. If the checksum fails, the receiving end can request a retransmission or perform error hiding based on historical metadata.

[0012] In a preferred embodiment, step S4, which involves scheduling the data transmission of the coordinated data packets and receiving confidence feedback from the cloud after identifying the transmitted data, specifically comprises the following steps: The collaborative controller first extracts the transmission priority of each region defined by the metadata of the first partition from the collaborative data packets to be transmitted, and assigns different network transmission service quality levels to the collaborative data packets of different regions according to the priority. At the same time, the collaborative controller monitors three types of status information in real time, including network transmission status, encoding efficiency status, and identification confidence feedback returned by the cloud. The network transmission status is calculated from the transmission and reception information of the collaborative data packets, including the average end-to-end latency and packet loss rate. The encoding efficiency status is obtained from the log information recorded during the encoding process in step S3, which includes the actual encoding bitrate and the average quantization parameter values ​​of the key regions. The identification confidence feedback is a confidence score between zero and one returned by the cloud for the identified key regions or complete video frames. Based on the aforementioned network transmission status, coding efficiency status, and recognition confidence feedback, the collaborative controller calculates a system performance index. The calculation process for the system performance index is as follows: the recognition confidence score is used as the numerator, divided by a penalty term to obtain a first intermediate value, and then the first intermediate value is multiplied by the logarithm of an efficiency term to obtain the final system performance index. The penalty term is calculated as: a number plus a delay penalty coefficient multiplied by the average end-to-end delay, plus a packet loss penalty coefficient multiplied by the packet loss rate. The efficiency term is calculated as: a number plus an efficiency coefficient multiplied by the sum of transmission priorities in all areas, divided by the actual coding rate.

[0013] In a preferred embodiment, the process of dynamically adjusting the network parameters of the lightweight neural network in step S1 and the transmission path decision of the bit rate mapping rule or cooperative data packet in step S3 based on confidence feedback specifically involves: The collaborative controller maintains a decision model, which is specifically a lightweight reinforcement learning policy network. The input state space of the decision model consists of a set of historical state vectors composed of the network transmission state, coding efficiency state and system performance index at the current time and multiple consecutive time steps before, as well as an estimate of the available bandwidth of the current network. The output action space of the decision model is defined as the joint adjustment of three types of parameters, including: the adjustment direction and fine-tuning magnitude of the key adjustable layer parameters of the lightweight neural network in step S1, namely the channel importance weight vector, spatial weight matrix, and convolutional layer parameters; the adjustment value of the bit rate mapping rule in step S3, specifically including the adjustment of the basic quantization parameter offset value allocated to different region type identifiers in the basic offset allocation sub-rule, the adjustment of the modulation intensity coefficient in the nonlinear modulation sub-rule, and the adjustment of relevant parameters of the global constraint sub-rule; and the decision on the cooperative data packet transmission path. The learning and decision-making mechanism of the decision-making model is as follows: the system performance index is used as the immediate reward obtained by the model decision; during the model training process, the sliding average of the system performance index over a period of time is calculated as the baseline reward; during the running phase, the policy network outputs a set of specific adjustment actions based on the current input state. The decision model is trained and updated using a combination of offline pre-training and online fine-tuning. After receiving the adjustment action, parameter adjustment and path decision are performed: For step S1, the parameter adjustment amounts output by the decision model for the channel importance weight vector, spatial weight matrix, and convolutional layer are applied to the lightweight neural network on the edge side using the learning rate for online fine-tuning to update its network parameters; For step S3, the adjustment values ​​output by the decision model for the bitrate mapping rules are used to directly update the corresponding parameters of the basic offset allocation sub-rule, nonlinear modulation sub-rule, and global constraint sub-rule in the video encoder configuration; For the transmission path, the routing table or link aggregation strategy of the network stack is updated according to the path decision output by the decision model.

[0014] The beneficial effects of this invention are as follows: By analyzing video frames in real time using a lightweight neural network, key regions and features in industrial product images are accurately located and extracted. A dynamic semantic scene graph is constructed to understand the content and prioritize it. The core of this invention lies in performing non-uniform intelligent encoding on the video stream based on the semantic importance of the content. While ensuring high-fidelity transmission of key information such as defects, non-critical regions are significantly compressed to save bandwidth. The transmission process is scheduled according to priority, and confidence feedback based on cloud recognition results is introduced to form a closed-loop control, thereby adaptively optimizing the front-end analysis model, encoding strategy, and network path. This enables continuous and stable low-latency, high-reliability visual quality inspection data transmission and accurate recognition in complex industrial network environments, significantly improving the overall efficiency and intelligence level of online quality inspection. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0018] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0019] Example 1 This embodiment provides, for example Figure 1 The image recognition-driven collaborative transmission method for industrial quality inspection data includes the following steps: Step S1: Analyze the acquired video frames in real time using a lightweight neural network deployed on an industrial camera or edge computing node to generate a first visual importance heatmap representing the salience of key regions, and extract the corresponding first intrinsic feature descriptors for the key regions. Step S2: Based on the first visual importance heatmap and the first intrinsic feature descriptor, construct a dynamically updated semantically enhanced MPEG-4 scene graph. The semantically enhanced MPEG-4 scene graph includes node attributes, spatiotemporal relationships, and outputs first partition metadata containing region type identifier and transmission priority. Step S3: Based on the first partition metadata output by the semantically enhanced MPEG-4 scene graph, and based on the preset bitrate mapping rules, perform non-uniform bitrate distribution encoding on the video frames. The key areas identified by the first partition metadata are encoded with low compression ratio, while the non-key areas are encoded with high compression ratio or skipped encoding. A collaborative data packet containing the encoded video basic stream and the first partition metadata is generated. Step S4: The collaborative controller schedules the data transmission of the collaborative data packet according to the transmission priority defined by the first partition metadata in the collaborative data packet, and receives the confidence feedback returned by the cloud after identifying the transmission data. Based on the confidence feedback, the network parameters of the lightweight neural network in step S1, the bit rate mapping rules in step S3, or the transmission path decision of the collaborative data packet are dynamically adjusted to form a closed-loop optimization.

[0020] In this embodiment, it should be specifically noted that in step S1, the real-time analysis of video frames by the lightweight neural network includes multi-scale feature fusion and spatial attention weighting, specifically as follows: The lightweight neural network employs a cascaded dilated convolutional pyramid structure. By setting dilated convolutional layers with different dilation rates in parallel, it simultaneously extracts depth feature maps of the input video frame under three different receptive fields, denoted as the first-scale feature map, the second-scale feature map, and the third-scale feature map, to capture defect features of different sizes. The three dilation rates can be set to 1, 2, and 4, respectively, with a corresponding convolutional kernel size of 3x3, to achieve a balance between computational cost and receptive field size. Subsequently, the first-scale, second-scale, and third-scale feature maps are fused through a channel-spatial dual attention unit. The channel attention mechanism first performs global average pooling on each first-scale, second-scale, or third-scale feature map, and then connects it through a fully connected layer. The layer generates importance weight vectors for feature channels, which are used to recalibrate feature channels and highlight channels related to defects. The fully connected layer may contain a bottleneck structure, which first compresses the number of channels to one-quarter of the original number and then restores it to the original number of channels to enhance nonlinearity and reduce parameters. The spatial attention mechanism is based on the spatial context relationship of the first-scale feature map, the second-scale feature map, and the third-scale feature map. It generates a spatial weight matrix with the same resolution as the first-scale feature map, the second-scale feature map, or the third-scale feature map through a convolutional layer. The high-weight positions correspond to potential key regions in the image that have complex textures or clear edges. The convolutional layer used to generate the spatial weight matrix may contain a 7x7 convolutional kernel to obtain broader contextual information for weight judgment. Next, the first-scale feature map, the second-scale feature map, and the third-scale feature map, which have been recalibrated by the channel importance weight vector, are multiplied element-wise with the spatial weight matrix to achieve weighted fusion and obtain a unified feature map. This element-wise multiplication operation enables the network to adaptively focus on the features most relevant to the defects in both the channel and spatial dimensions. Based on a unified feature map, a 1x1 convolutional layer is used to reduce the feature channel dimension. A Sigmoid activation function is applied to map each pixel value in the processed result to a closed interval between zero and one, thereby generating a single-channel first visual importance heatmap with the same spatial resolution as the input video frame. The value of each pixel in this first visual importance heatmap represents its visual saliency probability of belonging to a key region; this value is the saliency probability value of that pixel. The 1x1 convolutional layer reduces the number of channels to 1, and the Sigmoid function converts the scalar value at each position into a probability value. In the final output heatmap, regions with values ​​higher than a preset threshold (e.g., 0.5) can be considered as preliminary key region candidates. The channel importance weight vector, spatial weight matrix, and parameters of the convolutional layer together constitute the network parameters of the lightweight neural network. The specific process for extracting the first intrinsic feature descriptor for the key region is as follows: First, an adaptive threshold segmentation algorithm is applied to the first visual importance heatmap. The segmentation threshold is dynamically calculated based on the overall distribution of the first visual importance heatmap. Pixels with a saliency probability value higher than the threshold in the first visual importance heatmap are classified as key pixels, forming a binary mask. In the binary mask, all connected key regions are identified through connected component analysis. Each key region represents a continuous potential defect or main part of the product. The adaptive threshold segmentation uses Otsu's method to automatically determine the threshold to adapt to changes in image contrast. Through connected component analysis, noise regions with too small an area (e.g., less than 10 pixels) can be filtered out to ensure the continuity of key regions. Then, for each key region, within the corresponding spatial range of the unified feature map generated by the lightweight neural network in step S1, a region feature aggregation operation is performed to calculate the first intrinsic feature descriptor of the key region. Its value is obtained by multiplying the feature vector corresponding to each pixel in the unified feature map within the key region by the saliency probability value of that pixel in the first visual importance heatmap, summing all the resulting product feature vectors to form a summed vector; then, this summed vector is divided by a normalization factor, which is the first visual importance probability value of all pixels within the key region. The sum of the saliency probability values ​​in the importance heatmap, plus a positive constant to prevent division by zero; the calculation process is equivalent to taking a weighted average of the feature vectors at all locations within the key region, weighted by the saliency probability values. This makes the final first intrinsic feature descriptor mainly dominated by the visual features of the high saliency region, which has a smoothing effect on feature changes within the region and enhances its robustness to illumination and small deformations; adaptive threshold segmentation uses Otsu's method to automatically determine the threshold; the positive constant is set to the order of 10 to the power of negative 6; the dimension of the first intrinsic feature descriptor is consistent with the number of channels in the unified feature map.

[0021] In this embodiment, the specific process of constructing the dynamically updated semantically enhanced MPEG-4 scene graph in step S2 is as follows: First, based on the first visual importance heatmap and the first intrinsic feature descriptor generated in step S1, node attribute processing is performed on each key region. Node attributes include spatial attributes, semantic attributes, and dynamic weights, with the dynamic weights representing node importance. Spatial attributes are determined as follows: the abscissa of each pixel within the key region is calculated and multiplied by the saliency probability value of that pixel in the first visual importance heatmap; all multiplication results are summed to obtain a weighted sum of the abscissas. The ordinate of each pixel within the key region is also calculated and multiplied by the saliency probability value of that pixel in the first visual importance heatmap. The weighted sum of all products is obtained by summing the results of the products. The weighted sum of the x-coordinates and the weighted sum of the y-coordinates are then divided by the sum of the saliency probabilities of all pixels within the key region. The two quotients are used as the x-coordinate and y-coordinate of the center position of the region, respectively. The semantic attributes are obtained by classifying the first intrinsic feature descriptors using a lightweight classifier, which is used to identify the region type. The lightweight classifier can be a support vector machine or a micro neural network with a fully connected layer. The value of the dynamic weight is the largest saliency probability value among all pixels contained in the key region in the first visual importance heatmap. Secondly, semantic layer nodes and virtual coordination nodes are added to the standard MPEG-4 scene graph structure. Semantic layer nodes are used to characterize the processing status or defect severity level. Their attribute values ​​are dynamically generated by a small neural network trained on historical quality inspection data based on the current semantically enhanced MPEG-4 scene graph status. The small neural network can be a multilayer perceptron, and its input is the attribute aggregation features of all nodes in the current scene graph. Virtual coordination nodes are used to aggregate the shared features of multiple physical key area nodes in the same processing process to support the collaborative judgment of scattered defects. A spatiotemporal correlation is established by constructing cross-frame edges as a concrete manifestation of the spatiotemporal correlation through the calculation of feature similarity between nodes in adjacent frames. The feature similarity is calculated as follows: first, the first Mahalanobis distance component is calculated, which is based on the difference between the first intrinsic feature descriptors corresponding to adjacent frame nodes and the inverse of a feature covariance matrix; then, the second cosine similarity component is calculated, which is the dot product of two first intrinsic feature descriptors divided by the product of their respective moduli; finally, the result of transforming the first Mahalanobis distance component through a negative exponential function is multiplied by the second cosine similarity component to obtain the final feature similarity. If the feature similarity exceeds a preset threshold, a bidirectional edge is established, which indicates that two nodes in adjacent frames correspond to the same tracked target. The preset threshold can be set according to the experimental results of actual application scenarios, for example, 0.8. Finally, the semantically enhanced MPEG-4 scene graph is dynamically updated using an incremental graph convolutional network to optimize the relationships between node attributes and cross-frame edges. The dynamic update mechanism is as follows: when a new frame is added to the acquired video frame, for the nodes in the semantically enhanced MPEG-4 scene graph that need to be updated, their updated attribute representations are calculated. The calculation process for this attribute representation is as follows: for each neighboring node of the given node, the attribute representation of that neighboring node at the current time is taken, multiplied by a trainable weight matrix, and then multiplied by a normalization coefficient. This normalization coefficient is the product of the square root of the size of the neighbor set of the given node and its neighboring nodes. Then, all neighboring nodes... The product results of these points are summed, and finally a non-linear activation function, such as the ReLU function, is applied to the sum. This update process is only performed on newly added nodes, nodes with changed attributes, and their neighboring nodes in the semantically enhanced MPEG-4 scene graph, achieving local recomputation to reduce overhead. The updated semantically enhanced MPEG-4 scene graph output contains the first partition metadata, which includes the region type identifier and transmission priority. The transmission priority is calculated by weighting dynamic weights and a temporal stability factor. The temporal stability factor is quantified by the degree of difference in the first intrinsic feature descriptors corresponding to the same node across multiple frames. The specific process for outputting the first partition's metadata, which includes the region type identifier and transmission priority, is as follows: Based on the node attributes of the semantically enhanced MPEG-4 scene graph, the region type identifier is classified into critical defect region, product main body region, or background region. The classification is based on the output of the lightweight classifier in the node semantic attributes and is verified by combining the relative position of the key region in the spatial attributes. The verification can be based on the center position of the key region in the picture, the proportion of the pixel area it occupies, and its positional relationship with the preset reference region (such as the center of the picture or the product template region). For example, if the center of a region classified as "product main body region" is significantly deviated from the historical reference position, it may be corrected to "background region". Transmission priority is calculated using a linear weighted formula. The specific calculation process is as follows: A weighted sum is calculated. The first term of this sum is the dynamic weight multiplied by a first weight coefficient, and the second term is the temporal stability factor multiplied by a second weight coefficient. The sum of the first and second weight coefficients is one. The first weight coefficient can be 0.7, and the second weight coefficient is 0.3. The temporal stability factor is calculated as follows: For the same node with a tracking relationship established through bidirectional edges, calculate the Euclidean distance between the node's first intrinsic feature descriptor in the current frame and the corresponding first intrinsic feature descriptors in previous consecutive frames, obtaining multiple distance values. The number of consecutive frames can be preset, for example, 5 frames. Calculate the arithmetic mean of these distance values. Finally, subtract this arithmetic mean from one; the result is the temporal stability factor. The closer the factor value is to 1, the more stable the target features corresponding to the node are across multiple frames, and the higher its importance to the recognition task may be. The metadata of the first partition is stored in a structured form, including a unique identifier for the region, a region type identifier, transmission priority, and spatial coordinate information. After the metadata of the first partition is generated, it works in conjunction with the encoding module in step S3 to schedule the transmission order of data packets by transmission priority and to allocate a low compression ratio encoding strategy to high-priority regions to ensure that critical data is transmitted first when network resources are limited.

[0022] In this embodiment, it is particularly important to explain step S3, where the video frames are encoded using a non-uniform bitrate allocation based on a preset bitrate mapping rule. The bitrate mapping rule consists of three sub-rules: basic offset allocation, nonlinear modulation, and global constraints. The specific process is as follows: First, execute the basic offset allocation sub-rule: Based on the region type identifier in the first partition metadata, allocate a basic quantization parameter offset to each critical region; specifically: when the region type identifier is a critical defect region, the basic quantization parameter offset allocated to this region is a first negative integer, such as -12; when the region type identifier is a product main region, the basic quantization parameter offset allocated to this region is a second negative integer, the absolute value of which is less than the absolute value of the first negative integer, such as -6; when the region type identifier is a background region, the basic quantization parameter offset allocated to this region is a positive integer, such as positive 10. Secondly, nonlinear modulation sub-rules are executed: based on the transmission priority in the metadata of the first partition, the basic quantization parameter offset is nonlinearly modulated; the higher the transmission priority, the more the final quantization parameter offset after modulation tends to be adjusted negatively, and the adjustment magnitude increases with the increase of the transmission priority; a smoothing function is introduced during the modulation process to ensure a natural transition of quantization parameters between adjacent areas and avoid sudden changes in image quality; the smoothing function can use a Gaussian kernel function to perform a weighted average of the final quantization parameter offsets of adjacent areas; Finally, the global constraint sub-rules are executed: based on the network bandwidth budget, the overall bitrate of the current video frame to be encoded is constrained for compliance; after generating the quantization parameter mapping table using the basic offset allocation sub-rules and nonlinear modulation sub-rules, the overall bitrate to be generated by encoding according to the quantization parameter mapping table is predicted; if the predicted overall bitrate exceeds the target threshold, a gradient redistribution strategy is initiated, the specific process of which is as follows: a global quantization parameter adjustment amount is introduced, and based on the transmission priority in the first partition metadata, a local adjustment amount affected by the priority is calculated for each key region; the local adjustment amount is equal to the global quantization parameter adjustment amount, multiplied by an attenuation coefficient, and then multiplied by the difference obtained by subtracting the transmission priority of the region; among which, the higher the transmission priority of the region, the smaller the difference after subtracting the transmission priority, and the smaller the local adjustment amount it receives, thus receiving more protection in global compression; the final quantization parameter value used for encoding in each key region is obtained by adding the global basic quantization parameter value configured by the video encoder, the final quantization parameter offset obtained by the nonlinear modulation sub-rules for that region, and the local adjustment amount; the constrained quantization parameter mapping table is sent to the encoder to drive the partition differential encoding of the video frame; The specific operation of the nonlinear modulation sub-rule is as follows: The final quantization parameter offset for each critical region is calculated. Its value is the base quantization parameter offset for that region plus an adjustment term. The adjustment term is calculated as follows: First, an intermediate value is calculated, which is equal to the transmission priority of that region minus 0.5. Then, a negative exponential function with a base of the natural constant is applied to this intermediate value. The result is added to 1, the reciprocal is taken, and then 0.5 is subtracted to obtain a modulation factor between -0.5 and +0.5. Next, this modulation factor is multiplied by a preset modulation intensity coefficient, for example, 1.5, and then multiplied by the absolute value of the base quantization parameter offset for that region. The result is the adjustment term. The exponential coefficient in the negative exponential function controls how quickly the modulation factor changes with the transmission priority. This exponential coefficient is called the kurtosis coefficient of the modulation curve. The kurtosis coefficient is set to, for example, 6, to achieve rapid changes in the high-sensitivity range and saturation in the extreme value range. Furthermore, constraint weights based on regional spatial continuity are introduced. If the current region and the adjacent region belong to the same semantic object and have similar transmission priorities, the constraint weights will cause their final quantization parameter offsets to become consistent in order to maintain visual consistency within the object. The constraint weights can be calculated based on the difference between the spatial distance between the centers of the two regions and the transmission priority. The closer the distance and the smaller the priority difference, the greater the weight. The nonlinear modulation process integrates a dynamic sensitivity mechanism: by setting the kurtosis coefficient of the modulation curve to a value greater than 1, the modulation factor is highly sensitive to priority changes when the transmission priority is in the middle range, such as between 0.3 and 0.7; when the transmission priority is close to extremely high, such as greater than 0.9 or extremely low, such as less than 0.1, the sensitivity of the modulation factor to priority changes decreases, thereby avoiding excessive oscillations in the bit rate allocation due to small fluctuations in priority, and enhancing the stability of the system in dynamic environments; finally, the quantization parameter offset in all regions is limited to a preset range, ensuring stable encoder operation. The process of generating a cooperative data packet containing the encoded video elementary stream and the metadata of the first partition is as follows: The video encoder compresses video frames based on a quantization parameter mapping table used to drive differential coding of video frame partitions, outputting a video elementary stream. The video elementary stream is then encapsulated with first partition metadata to generate a co-occurrence data packet. This encapsulation process ensures that the video elementary stream and the first partition metadata are time-aligned. Specifically, the first partition metadata corresponding to the video frame is written into the supplemental enhancement information region of the video elementary stream, or into the header of a custom video container format, according to the timestamp or decoding order of the video frame. In the co-occurrence data packet, the spatial coordinate information contained in the first partition metadata is mapped to the position index of the block structure or macroblock used during video elementary stream encoding. This allows the receiving end of the co-occurrence data packet to accurately associate the region type identifier and transmission priority information carried in the first partition metadata with the specific spatial region in the video content based on the mapping relationship. The block structure can refer to the slice or tile structure in the video coding standard. The multiplexed structure of the cooperative data packet includes a synchronization header and a timestamp to ensure that the receiving end can accurately parse and reconstruct the data stream. The first partition metadata in the cooperative data packet embeds a checksum to detect transmission errors. If the checksum fails, the receiving end can request retransmission or perform error hiding based on historical metadata. The format design of the cooperative data packet supports extension, allowing the embedding of new semantic information or control instructions in the future to adapt to diverse quality inspection needs.

[0023] In this embodiment, the process of scheduling the data transmission of the coordinated data packet and receiving the confidence feedback returned by the cloud after identifying the transmitted data in step S4 is specifically as follows: The collaborative controller first extracts the transmission priority of each region defined by the metadata of the first partition from the collaborative data packets to be transmitted, and assigns different network transmission service quality levels to the collaborative data packets of different regions according to the priority. The higher the priority of a region, the greater the scheduling weight of its data packets in the network queue, and the higher the transmission latency guarantee level. At the same time, the collaborative controller monitors three types of status information in real time, including network transmission status, encoding efficiency status, and identification confidence feedback returned by the cloud. The network transmission status is calculated from the transmission and reception information of the collaborative data packets, including the average end-to-end latency and packet loss rate. The encoding efficiency status is obtained from the log information recorded during the encoding process in step S3, which includes the actual encoding bitrate and the average quantization parameter values ​​of the key regions. The identification confidence feedback is a confidence score between zero and one returned by the cloud for the identified key regions or complete video frames. Based on the network transmission status, coding efficiency status, and recognition confidence feedback, the cooperative controller calculates a system performance index. The calculation process for the system performance index is as follows: the recognition confidence score is used as the numerator, divided by a penalty term to obtain a first intermediate value, and then the first intermediate value is multiplied by the logarithm of an efficiency term to obtain the final system performance index. The penalty term is calculated as: a number plus a delay penalty coefficient multiplied by the average end-to-end delay, plus a packet loss penalty coefficient multiplied by the packet loss rate. The efficiency term is calculated as: a number plus an efficiency coefficient multiplied by the sum of transmission priorities of all areas, divided by the actual coding rate. The delay penalty coefficient, packet loss penalty coefficient, and efficiency coefficient are all preset positive real configurable parameters. The delay penalty coefficient can be selected between 0.1 and 0.5, the packet loss penalty coefficient can be selected between 0.05 and 0.2, and the efficiency coefficient can be selected between 0.1 and 0.3. The system performance index is used to quantify the overall task execution performance of the system under the current parameter configuration and given network and computing resources, and serves as a unified evaluation basis for closed-loop optimization; The process of dynamically adjusting the network parameters of the lightweight neural network in step S1 and the transmission path decision of the bit rate mapping rule or cooperative data packet in step S3 based on confidence feedback is as follows: The collaborative controller maintains a decision model, which is specifically a lightweight reinforcement learning policy network. The input state space of the decision model consists of a set of historical state vectors composed of the network transmission state, coding efficiency state and system performance index at the current time and multiple consecutive time steps before, as well as an estimate of the available bandwidth of the current network. The length of the sliding time window corresponding to "multiple consecutive time steps" can be set according to the system response speed requirements, such as ten to thirty frames. The output action space of the decision model is defined as the joint adjustment of three types of parameters, including: the adjustment direction and fine-tuning magnitude of the key adjustable layer parameters of the lightweight neural network in step S1, namely the channel importance weight vector, spatial weight matrix, and convolutional layer parameters; the adjustment value of the bit rate mapping rule in step S3, specifically including the adjustment of the basic quantization parameter offset value allocated to different region type identifiers in the basic offset allocation sub-rule, the adjustment of the modulation intensity coefficient in the nonlinear modulation sub-rule, and the adjustment of relevant parameters of the global constraint sub-rule; and the decision on the cooperative data packet transmission path, which is to select or allocate the transmission ratio among multiple optional physical or logical network transmission paths. The learning and decision-making mechanism of the decision model is as follows: the system performance index is used as the immediate reward obtained by the model decision; during the model training process, the moving average of the system performance index over a period of time is calculated as the baseline reward to assist in the calculation of the advantage function and reduce training variance; the time period for calculating the moving average can be consistent with the time window length of the input state of the decision model; during the running phase, the policy network outputs a set of specific adjustment actions based on the current input state, and this set of adjustment actions is applied to the data processing and transmission of the next time period. The training and updating of the decision model adopts a combination of offline pre-training and online fine-tuning: In the initial stage, the policy network is pre-trained using historical running data collected from historical quality inspection tasks or a simulation environment built by simulating real networks and quality inspection scenarios; after deployment and operation, data sequences consisting of states, actions and rewards are continuously collected, and the internal parameters of the policy network are updated periodically using the policy gradient optimization algorithm, so that the model learns how to adjust the front-end parameters under different network states and recognition quality trends to maximize the long-term accumulated system performance indicators; After receiving the adjustment action, parameter adjustment and path decision are performed: For step S1, the parameter adjustment amounts output by the decision model for the channel importance weight vector, spatial weight matrix, and convolutional layer are applied to the lightweight neural network on the edge side with a small learning rate for online fine-tuning to update its network parameters; the small learning rate can be set on the order of one ten-thousandth to one thousandth; For step S3, the adjustment values ​​output by the decision model for the bitrate mapping rules are used to directly update the corresponding parameters of the basic offset allocation sub-rule, nonlinear modulation sub-rule, and global constraint sub-rule in the video encoder configuration; For the transmission path, the routing table or link aggregation strategy of the network stack is updated according to the path decision output by the decision model. While performing adjustments, a stability assurance mechanism is activated: First, an absolute value upper limit constraint is applied to the parameter adjustment amounts applied to steps S1 and S3 to prevent excessive single adjustment from causing system performance fluctuations; the absolute value upper limit can be set to 0.1; Second, freeze and reset conditions are set. When a sharp drop in system performance indicators is detected for multiple consecutive frames, such as a drop of more than 30% within five frames, or when the recognition confidence score returned from the cloud is lower than a preset danger threshold, such as lower than 0.3, all parameter adjustments are temporarily frozen, and the relevant parameters of step S1 or step S3 are rolled back to the most recently saved stable version, while triggering a system alarm; Third, the parameter adjustments of the lightweight neural network in step S1 adopt a gradual parameter update method during the fine-tuning process to retain its main performance for historical tasks and avoid catastrophic forgetting of the features already mastered due to learning new data.

[0024] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0025] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0026] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0027] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0028] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0029] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0030] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for collaborative transmission of industrial quality inspection data driven by image recognition, characterized in that, Specifically, the steps include the following: Step S1: Analyze the acquired video frames in real time using a lightweight neural network deployed on an industrial camera or edge computing node to generate a first visual importance heatmap representing the salience of key regions, and extract the corresponding first intrinsic feature descriptors for the key regions. Step S2: Based on the first visual importance heatmap and the first intrinsic feature descriptor, construct a dynamically updated semantically enhanced MPEG-4 scene graph. The semantically enhanced MPEG-4 scene graph includes node attributes, spatiotemporal relationships, and outputs first partition metadata containing region type identifier and transmission priority. Step S3: Based on the first partition metadata output by the semantically enhanced MPEG-4 scene graph, and based on the preset bitrate mapping rules, perform non-uniform bitrate distribution encoding on the video frames. The key areas identified by the first partition metadata are encoded with low compression ratio, while the non-key areas are encoded with high compression ratio or skipped encoding. A collaborative data packet containing the encoded video basic stream and the first partition metadata is generated. Step S4: The collaborative controller schedules the data transmission of the collaborative data packet according to the transmission priority defined by the first partition metadata in the collaborative data packet, and receives the confidence feedback returned by the cloud after identifying the transmission data. Based on the confidence feedback, the network parameters of the lightweight neural network in step S1, the bit rate mapping rules in step S3, or the transmission path decision of the collaborative data packet are dynamically adjusted to form a closed-loop optimization.

2. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 1, characterized in that: In step S1, the real-time analysis of video frames by the lightweight neural network includes multi-scale feature fusion and spatial attention weighting, specifically: The lightweight neural network adopts a cascaded dilated convolutional pyramid structure. By setting dilated convolutional layers with different dilation rates in parallel, it simultaneously extracts the depth feature maps of the input video frame under three different receptive fields, which are denoted as the first-scale feature map, the second-scale feature map, and the third-scale feature map, respectively. Subsequently, the first-scale feature map, the second-scale feature map, and the third-scale feature map are fused through a dual channel-space attention unit. The channel attention mechanism first performs global average pooling on each first-scale feature map, second-scale feature map, or third-scale feature map, and generates an importance weight vector for the feature channel through a fully connected layer for recalibrating the feature channel. The spatial attention mechanism, based on the spatial context relationship of the first-scale feature map, second-scale feature map, and third-scale feature map, generates a spatial weight matrix with the same resolution as the first-scale feature map, second-scale feature map, or third-scale feature map through a convolutional layer. Next, the first-scale feature map, the second-scale feature map, and the third-scale feature map, which have been recalibrated by the channel importance weight vector, are multiplied element-wise with the spatial weight matrix to achieve weighted fusion and obtain a unified feature map. Based on a unified feature map, a convolutional layer with a kernel size of 1x1 is used for processing, and a sigmoid activation function is applied to map each pixel value of the processing result to a closed interval between zero and one, thereby generating a single-channel first visual importance heatmap with the same spatial resolution as the input video frame. The value of each pixel in the first visual importance heatmap represents the visual saliency probability of its belonging to a key region, and this value is the saliency probability value of the pixel. The channel importance weight vector, spatial weight matrix, and parameters of the convolutional layer together constitute the network parameters of the lightweight neural network.

3. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 2, characterized in that: The specific process for extracting the first intrinsic feature descriptor corresponding to the key region is as follows: First, an adaptive threshold segmentation algorithm is applied to the first visual importance heatmap. The segmentation threshold is dynamically calculated based on the overall distribution of the first visual importance heatmap. Pixels with a saliency probability value higher than the threshold in the first visual importance heatmap are classified as key pixels, forming a binary mask. In the binary mask, all connected key regions are identified through connected component analysis. Each key region represents a continuous potential defect or main part of the product. Then, for each key region, within the corresponding spatial range of the unified feature map generated by the lightweight neural network in step S1, a region feature aggregation operation is performed to calculate the first intrinsic feature descriptor of the key region. Its value is: multiplying the feature vector corresponding to each pixel in the unified feature map within the key region by the saliency probability value of the pixel in the first visual importance heatmap, and summing all the product feature vectors to form a sum vector; then dividing this sum vector by a normalization factor, which is the sum of the saliency probability values ​​of all pixels in the key region in the first visual importance heatmap, plus a positive constant.

4. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 3, characterized in that: In step S2, the specific process of constructing the dynamically updated semantically enhanced MPEG-4 scene graph is as follows: First, based on the first visual importance heatmap and the first intrinsic feature descriptor generated in step S1, each key region is subjected to node attribute processing, where node attributes include spatial attributes, semantic attributes, and dynamic weights. The spatial attributes are determined as follows: the abscissa of each pixel in the key region is calculated by multiplying it by the saliency probability value of that pixel in the first visual importance heatmap, and all products are summed to obtain a weighted sum of abscissas; the ordinate of each pixel in the key region is calculated by multiplying it by the saliency probability value of that pixel in the first visual importance heatmap, and all products are summed to obtain a weighted sum of ordinates; the weighted sum of abscissas and the weighted sum of ordinates are divided by the sum of the saliency probability values ​​of all pixels in the key region, and the two quotients are used as the abscissa and ordinate of the center position of the region, respectively; the semantic attributes are obtained by classifying the first intrinsic feature descriptor using a lightweight classifier and are used to identify the region type; the value of the dynamic weight is the largest saliency probability value in the first visual importance heatmap among all pixels contained in the key region. Secondly, semantic layer nodes and virtual coordination nodes are added to the standard MPEG-4 scene graph structure; the semantic layer nodes are used to characterize the processing procedure status or defect severity level; the virtual coordination nodes are used to aggregate the shared features of multiple physical key area nodes in the same processing procedure. A spatiotemporal correlation is established by constructing cross-frame edges that represent the spatiotemporal correlation through the calculation of feature similarity between adjacent frame nodes. The feature similarity is calculated as follows: first, a first Mahalanobis distance component is calculated, which is based on the difference between the first intrinsic feature descriptors corresponding to adjacent frame nodes and the inverse of a feature covariance matrix; then, a second cosine similarity component is calculated, which is the dot product of the two first intrinsic feature descriptors divided by the product of their respective moduli; finally, the result of transforming the first Mahalanobis distance component through a negative exponential function is multiplied by the second cosine similarity component to obtain the final feature similarity. If the feature similarity exceeds a preset threshold, a bidirectional edge is established. Finally, the semantically enhanced MPEG-4 scene graph is dynamically updated by using an incremental graph convolutional network to optimize the relationship between node attributes and the edges represented by the cross-frame edges. The dynamic update mechanism is as follows: when a new frame is added to the acquired video frame, for the nodes in the semantically enhanced MPEG-4 scene graph that need to be updated, the updated attribute representation is calculated. The calculation process of the attribute representation is as follows: for each neighbor node of the node, the attribute representation of the neighbor node at the current time is taken, multiplied by a trainable weight matrix, and then multiplied by a normalization coefficient, which is the product of the square root of the size of the neighbor set of the node and its neighbor nodes. Then, the product results of all neighbor nodes are summed, and finally a non-linear activation function is applied to the summation result. This update process is only performed on newly added nodes, nodes whose attributes have changed, and their neighboring nodes in the semantically enhanced MPEG-4 scene graph; The updated semantically enhanced MPEG-4 scene graph output includes first partition metadata with region type identifier and transmission priority, wherein the transmission priority is calculated by weighting the dynamic weight and the timing stability factor.

5. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 4, characterized in that: The specific process by which the first partition metadata, including the region type identifier and transmission priority, is output is as follows: Based on the node attributes of the semantically enhanced MPEG-4 scene graph, the region type identifier is classified into critical defect region, product main body region or background region; the classification is based on the output of the lightweight classifier in the node semantic attributes and is verified by combining the relative position of the key region in the spatial attributes. The transmission priority is calculated using a linear weighted formula. The specific calculation process is as follows: A weighted sum is calculated, where the first term of the sum is the dynamic weight multiplied by a first weight coefficient, and the second term is the temporal stability factor multiplied by a second weight coefficient. The temporal stability factor is calculated as follows: For the same node that has established a tracking relationship through the bidirectional edge, the Euclidean distance between the node's first intrinsic feature descriptor in the current frame and the corresponding first intrinsic feature descriptors in previous consecutive frames is calculated, resulting in multiple distance values. The arithmetic mean of these distance values ​​is then calculated. Finally, this arithmetic mean is subtracted from the calculated value, and the result is the temporal stability factor. The metadata of the first partition is stored in a structured form, including a unique identifier for the region, a region type identifier, a transmission priority, and spatial coordinate information.

6. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 5, characterized in that: In step S3, the video frames are encoded using a non-uniform bitrate distribution based on a preset bitrate mapping rule. The bitrate mapping rule consists of three sub-rules: basic offset allocation, nonlinear modulation, and global constraints. The specific process is as follows: First, execute the basic offset allocation sub-rule: Based on the region type identifier in the first partition metadata, allocate a basic quantization parameter offset to each critical region; specifically: when the region type identifier is a critical defect region, the basic quantization parameter offset allocated to this region is a first negative integer; when the region type identifier is a product main body region, the basic quantization parameter offset allocated to this region is a second negative integer, the absolute value of which is less than the absolute value of the first negative integer; when the region type identifier is a background region, the basic quantization parameter offset allocated to this region is a positive integer. Secondly, execute the nonlinear modulation sub-rule: combine the transmission priority in the metadata of the first partition with the nonlinear modulation of the basic quantization parameter offset; Finally, the global constraint sub-rules are executed: based on the network bandwidth budget, the overall bitrate of the current video frame to be encoded is constrained for compliance; after generating the quantization parameter mapping table using the basic offset allocation sub-rules and nonlinear modulation sub-rules, the overall bitrate to be generated by encoding according to the quantization parameter mapping table is predicted; if the predicted overall bitrate exceeds the target threshold, the gradient redistribution strategy is initiated, the specific process of which is as follows: a global quantization parameter adjustment amount is introduced, and a local adjustment amount affected by the priority is calculated for each key region based on the transmission priority in the first partition metadata; the local adjustment amount is equal to the global quantization parameter adjustment amount, multiplied by an attenuation coefficient, and then multiplied by the difference obtained by subtracting the transmission priority of the region; the final quantization parameter value used for encoding in each key region is obtained by adding the global basic quantization parameter value configured by the video encoder, the final quantization parameter offset obtained by the nonlinear modulation sub-rules for that region, and the local adjustment amount; the constrained quantization parameter mapping table is sent to the encoder to drive the partition differential encoding of the video frame.

7. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 6, characterized in that: The specific operation of the nonlinear modulation sub-rule is as follows: The final quantization parameter offset for each critical region is calculated as follows: the base quantization parameter offset for that region plus an adjustment term. The adjustment term is calculated as follows: First, an intermediate value is calculated, which is equal to the transmission priority of that region minus 0.

5. Then, a negative exponential function with the natural constant as the base is applied to this intermediate value, the result is added by 1, the reciprocal is taken, and then 0.5 is subtracted to obtain a modulation factor. After that, this modulation factor is multiplied by a preset modulation intensity coefficient, and then multiplied by the absolute value of the base quantization parameter offset for that region. The result is the adjustment term.

8. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 7, characterized in that: The process of generating a cooperative data packet containing the encoded video elementary stream and the metadata of the first partition is as follows: The video encoder compresses video frames according to the quantization parameter mapping table used to drive the differential coding of video frame partitions and outputs the video basic stream; the video basic stream is encapsulated with the first partition metadata to generate a cooperative data packet; The multiplexed structure of the coordinated data packet includes a synchronization header and a timestamp. The first partition metadata in the coordinated data packet embeds a checksum to detect transmission errors. If the verification fails, the receiving end can request a retransmission or perform error hiding based on historical metadata.

9. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 8, characterized in that: In step S4, the process of scheduling the data transmission of the coordinated data packets and receiving the confidence feedback returned by the cloud after identifying the transmitted data is specifically as follows: The collaborative controller first extracts the transmission priority of each region defined by the metadata of the first partition from the collaborative data packets to be transmitted, and assigns different network transmission service quality levels to the collaborative data packets of different regions according to the priority; at the same time, the collaborative controller monitors three types of status information in real time, including network transmission status, encoding efficiency status, and identification confidence feedback returned by the cloud. The network transmission status is calculated from the information on the transmission and reception of collaborative data packets, including the average end-to-end latency and packet loss rate; the coding efficiency status is obtained from the log information recorded during the coding process in step S3, which includes the actual coding bitrate and the average quantization parameter value of the key area; the identification confidence feedback is the confidence score returned by the cloud for the identified key area or complete video frame, with a value between zero and one. Based on the aforementioned network transmission status, coding efficiency status, and recognition confidence feedback, the cooperative controller calculates a system performance index. The calculation process for the system performance index is as follows: the recognition confidence score is used as the numerator, divided by a penalty term to obtain a first intermediate value, and then the first intermediate value is multiplied by the logarithm of an efficiency term to obtain the final system performance index. The penalty term is calculated as: a number plus a delay penalty coefficient multiplied by the average end-to-end delay, plus a packet loss penalty coefficient multiplied by the packet loss rate. The efficiency term is calculated as: a number plus an efficiency coefficient multiplied by the sum of transmission priorities of all areas, divided by the actual coding rate.

10. The image recognition-driven collaborative transmission method for industrial quality inspection data according to claim 9, characterized in that: The process of dynamically adjusting the network parameters of the lightweight neural network in step S1 and the transmission path decision of the bit rate mapping rule or cooperative data packet in step S3 based on confidence feedback is as follows: The collaborative controller maintains a decision model, which is specifically a lightweight reinforcement learning policy network. The input state space of the decision model consists of a set of historical state vectors composed of the network transmission state, coding efficiency state and system performance index at the current time and multiple consecutive time steps before, as well as an estimate of the available bandwidth of the current network. The output action space of the decision model is defined as the joint adjustment of three types of parameters, including: the adjustment direction and fine-tuning magnitude of the key adjustable layer parameters of the lightweight neural network in step S1, namely the channel importance weight vector, spatial weight matrix, and convolutional layer parameters; the adjustment value of the bit rate mapping rule in step S3, specifically including the adjustment of the basic quantization parameter offset value allocated to different region type identifiers in the basic offset allocation sub-rule, the adjustment of the modulation intensity coefficient in the nonlinear modulation sub-rule, and the adjustment of relevant parameters of the global constraint sub-rule; and the decision on the cooperative data packet transmission path. The learning and decision-making mechanism of the decision-making model is as follows: the system performance index is used as the immediate reward obtained by the model decision; during the model training process, the sliding average of the system performance index over a period of time is calculated as the baseline reward; during the running phase, the policy network outputs a set of specific adjustment actions based on the current input state. The decision model is trained and updated using a combination of offline pre-training and online fine-tuning. After receiving the adjustment action, parameter adjustment and path decision are performed: For step S1, the parameter adjustment amounts output by the decision model for the channel importance weight vector, spatial weight matrix, and convolutional layer are applied to the lightweight neural network on the edge side using the learning rate for online fine-tuning to update its network parameters; For step S3, the adjustment values ​​output by the decision model for the bitrate mapping rules are used to directly update the corresponding parameters of the basic offset allocation sub-rule, nonlinear modulation sub-rule, and global constraint sub-rule in the video encoder configuration; For the transmission path, the routing table or link aggregation strategy of the network stack is updated according to the path decision output by the decision model.

Images