Edge computing inference method and related devices

Abstract

The application discloses an edge computing inference method and related equipment, and relates to the technical field of visual detection. The method comprises the following steps: acquiring a first time corresponding to an external trigger signal and a trigger identifier; acquiring a target frame image corresponding to the trigger identifier and a second time of the target frame image; if a time difference between the first time and the second time is less than or equal to a preset deviation threshold, performing first inference processing on the target frame image based on a first inference path; if the predicted total time consumption of the first inference processing is greater than a preset upper limit of inference time during the first inference processing, terminating the first inference processing, performing second inference processing on the target frame image based on a second inference path to obtain an inference result, otherwise, continuing to perform the first inference processing to obtain the inference result; and outputting an inference data packet. The application can improve the time determinacy, result reliability and data traceability of the edge computing inference process.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and more specifically, to an edge computing inference method and related equipment. Background Technology

[0002] With the continuous development of intelligent manufacturing, industrial visual inspection, and edge artificial intelligence technologies, edge computing-based visual inspection systems are increasingly being applied to scenarios such as automated optical inspection, intelligent quality inspection, and industrial robots due to their advantages of low latency, high real-time performance, and flexible on-site deployment. As an important technical means to achieve real-time perception and intelligent decision-making of industrial equipment, the stability and determinism of the edge computing inference process directly affect the inspection efficiency and reliability of the visual inspection system.

[0003] However, in related technologies, edge computing terminals typically need to execute multiple tasks simultaneously, such as image acquisition, model inference, and data transmission. These tasks share operating system scheduling resources, leading to time uncertainties between external trigger events and image acquisition and inference processing. Traditional methods usually reduce task latency by increasing cache capacity, adjusting task priorities, or optimizing scheduling strategies, but these methods struggle to fundamentally address the issue of a lack of strict correspondence between external trigger signals and inference tasks. Particularly in scenarios with multiple triggers, high-frequency acquisition, or limited computing resources, problems such as mismatches between image frames and trigger events, increased inference task waiting time, and reduced timeliness of inference results can easily arise. In other words, related technologies suffer from technical problems such as uncertain correspondence between external trigger events and inference tasks during edge computing inference, uncontrollable inference latency, and low reliability of inference results. Summary of the Invention

[0004] In the summary section of this application, the relevant technical solutions are described in general terms, and a series of simplified concepts are introduced. These concepts will be further elaborated in the detailed embodiments section. This summary section should not be construed as limiting the key or essential technical features of the claimed solutions, nor is it intended to limit the scope of protection of the claimed solutions.

[0005] The edge computing inference method and related equipment provided in this application can improve the temporal determinism, result reliability and data traceability of the edge computing inference process by establishing a deterministic association between external triggering events and target frame images, and by combining time sequence verification, dynamic switching of inference paths and inference data traceability mechanism.

[0006] In a first aspect, this application provides an edge computing inference method applied to an edge computing terminal, comprising: acquiring a first moment and a trigger identifier corresponding to an external trigger signal, wherein the external trigger signal is independent of the internal task scheduling clock of the edge computing terminal; acquiring a corresponding target frame image and a second moment of the target frame image based on the trigger identifier; if the time difference between the first moment and the second moment is less than or equal to a preset deviation threshold, performing a first inference process on the target frame image based on a first inference path; if, during the first inference process, the estimated total time consumption of the first inference process is greater than a preset inference duration upper limit, terminating the first inference process and performing a second inference process on the target frame image based on a second inference path to obtain an inference result; otherwise, continuing to execute the first inference process to obtain the inference result, wherein the computational resource consumption of the second inference process is lower than that of the first inference process; and outputting an inference data packet, the inference data packet including the trigger identifier, the first moment, the second moment, and the inference result.

[0007] In some implementations, obtaining the first moment and trigger identifier corresponding to the external trigger signal includes: obtaining the system timer count value when the trigger event occurs through a hardware interrupt service routine, and determining the system timer count value as the first moment; generating a globally unique sequence number as the trigger identifier based on the receiving order of the external trigger signal according to a preset incrementing rule.

[0008] In some implementations, obtaining the corresponding target frame image and the second moment of the target frame image based on the trigger identifier includes: retrieving the target frame image with the smallest time difference between the frame capture timestamp and the first moment from the frame buffer queue of the image acquisition device; and determining the frame capture timestamp of the target frame image as the second moment.

[0009] In some embodiments, the edge computing inference method further includes: if the time difference between the first time point and the second time point is greater than the preset deviation threshold, then acquiring the historical frame image corresponding to the previous trigger identifier adjacent to the trigger identifier; performing inter-frame difference processing on the target frame image and the historical frame image to obtain a difference image; determining whether the target frame image is a repeated trigger acquisition frame based on the distribution characteristics of non-zero pixels in the difference image; if it is determined to be a repeated trigger acquisition frame, then using the historical inference result corresponding to the historical frame image as the inference result of the target frame image, and generating an inference data packet containing a reuse identifier; if it is determined to be a non-repeated trigger acquisition frame, then generating a detection result containing an invalid state flag and the trigger identifier.

[0010] In some implementations, terminating the first inference process if the estimated total time of the first inference process exceeds a preset inference time limit includes: obtaining the number of inference layers currently completed by the preset artificial intelligence inference model and the total number of inference layers in the preset artificial intelligence inference model; determining the estimated remaining time to complete the remaining inference layers based on a first ratio of the number of completed inference layers to the total number of inference layers and the consumed inference time from the start of the first inference process to the current moment; obtaining the current operating temperature of the edge computing terminal and the current load rate of the neural network processor; correcting the estimated remaining time based on a first deviation between the current operating temperature and a preset reference temperature and a second deviation between the current load rate and a preset reference load rate to obtain a corrected estimated remaining time; and sending an inference termination command to the preset artificial intelligence inference model if the sum of the consumed inference time and the corrected estimated remaining time exceeds the preset inference time limit.

[0011] In some implementations, the second inference path includes inference processing based on a preset lightweight detection algorithm, which is deployed in an independent hardware acceleration unit in the edge computing terminal. The independent hardware acceleration unit performs resource locking during the system initialization phase of the edge computing terminal to prevent it from being preempted by other tasks during operation.

[0012] In some implementations, after the output inference data packet, the edge computing inference method further includes: obtaining the time difference between the first time and the second time in a plurality of inference data packets within a preset statistical period; determining the trigger-frame synchronization jitter distribution within the preset statistical period based on the plurality of time differences; and adjusting the preset deviation threshold and / or the preset inference duration upper limit based on the trigger-frame synchronization jitter distribution.

[0013] In some implementations, dynamically adjusting the preset deviation threshold and / or the preset inference duration upper limit based on the trigger-frame synchronization jitter distribution includes: acquiring product yield data of the production line corresponding to the edge computing terminal; constructing a jitter-yield correlation model based on the trigger-frame synchronization jitter distribution and the product yield data; predicting the expected yield under different inference duration configurations in the next statistical period according to the jitter-yield correlation model; and adjusting the preset inference duration upper limit to a target inference duration, wherein the target inference duration maximizes the expected yield while satisfying preset yield constraints.

[0014] In some implementations, the output inference data packet includes: obtaining the model version identifier, confidence information, degradation indication information, and image acquisition device identifier corresponding to the output of the first inference process or the second inference process; generating a data packet digest signature based on the unique device identifier and current timestamp of the edge computing terminal; and combining the model version identifier, the confidence information, the degradation indication information, the image acquisition device identifier, the data packet digest signature, the trigger identifier, the first time point, the second time point, and the inference result into the inference data packet.

[0015] Secondly, this application also provides an edge computing inference device applied to an edge computing terminal, comprising: a signal acquisition unit, configured to acquire a first moment and a trigger identifier corresponding to an external trigger signal, wherein the external trigger signal is independent of the internal task scheduling clock of the edge computing terminal; an image acquisition unit, configured to acquire a corresponding target frame image and a second moment of the target frame image based on the trigger identifier; a time difference processing unit, configured to perform a first inference processing on the target frame image based on a first inference path if the time difference between the first moment and the second moment is less than or equal to a preset deviation threshold; an inference processing unit, configured to terminate the first inference processing and perform a second inference processing on the target frame image based on a second inference path to obtain an inference result if the estimated total time of the first inference processing is greater than a preset inference duration upper limit during the first inference processing; otherwise, to continue executing the first inference processing to obtain the inference result, wherein the computational resource consumption of the second inference processing is lower than that of the first inference processing; and a data output unit, configured to output an inference data packet, the inference data packet including the trigger identifier, the first moment, the second moment, and the inference result.

[0016] Thirdly, this application also provides an electronic device, including: a memory and a processor, wherein the processor is configured to implement the steps of the edge computing inference method described in the first aspect when executing a computer program stored in the memory.

[0017] Fourthly, this application also provides a computer-readable storage medium storing computer-executable instructions or a computer program, which, when executed by a processor, implement the steps of the edge computing inference method described in the first aspect.

[0018] Fifthly, this application also provides a computer program product, including a computer program or computer executable instructions, wherein when the computer program or computer executable instructions are executed by a processor, the steps of the edge computing inference method provided in the embodiments of this application are implemented.

[0019] In summary, this application establishes a correspondence between external trigger events and image data by acquiring the first moment and trigger identifier corresponding to an external trigger signal that is independent of the internal task scheduling clock of the edge computing terminal, and acquiring the corresponding target frame image and second moment based on the trigger identifier. Since the external trigger signal is not affected by the internal task scheduling of the edge computing terminal, it avoids the problem of uncertainty in the correspondence between trigger events and image acquisition caused by factors such as operating system task scheduling and computing resource contention, thereby improving the time consistency between inference task input data and actual trigger events. By comparing the time difference between the first moment and the second moment, and only executing inference processing based on the first inference path when the time difference meets a preset deviation threshold, the temporal validity of the target frame image can be judged. This allows for the selection of target frame images that match the time of the external trigger event, reducing the problem of inaccurate inference results caused by excessive deviation between the trigger signal and the image acquisition time, and improving edge computing performance. The reliability of the computing terminal when executing inference tasks is improved. During the first inference process, by determining whether the estimated total time of the first inference process exceeds the preset inference time limit, and terminating the first inference process if it does, the process switches to a second inference path with lower computational resource consumption. This avoids the problem of complex inference tasks continuously consuming computational resources and thus failing to complete in a timely manner. Since the computational resource consumption of the second inference path is lower than that of the first inference path, inference results can continue to be output even when inference time is limited, thereby improving the time controllability of the inference process. By outputting an inference data packet containing the trigger identifier, the first moment, the second moment, and the inference result, the correlation information between the external trigger event, the image acquisition process, and the inference result can be preserved. This allows for subsequent tracing and analysis of the inference process based on the inference data packet. While ensuring the correspondence between the inference task and the external trigger event, it also improves the data integrity and traceability of the inference result. In summary, the edge computing inference method provided in this application improves the time determinism, result reliability, and data traceability of the edge computing inference process by establishing a deterministic association between the external trigger event and the target frame image, combined with timing verification, dynamic switching of the inference path, and an inference data traceability mechanism. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0021] Figure 1 A flowchart illustrating an edge computing inference method provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the composition structure of an edge computing inference device provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] The terms used in the specification, claims, and drawings of this application, such as "first," "second," "third," "fourth," etc. (if any), are used to distinguish similar objects and not to describe a specific order or sequence. Therefore, it is to be understood that these terms can be used interchangeably where appropriate, allowing the described embodiments to be used in different orders, unless specifically required by the illustrations or description. Furthermore, the terms "is" and "has," and any variations thereof, are intended to cover, non-exclusively, all possible constituent elements. For example, a process, method, system, product, or apparatus comprising several steps or units is not necessarily limited to the steps or units explicitly listed, but may also include other steps or units not explicitly listed, or steps or units inherent to the process, method, product, or apparatus.

[0025] In this application, a "module" or "unit" refers to a computer program or part of a computer program that has a specific function and works in conjunction with other related parts to achieve a predetermined goal. These modules or units can be implemented by software, hardware (e.g., processing circuitry or memory), or a combination of both. One or more processors or memories can implement one or more modules or units. Furthermore, each module or unit can also be part of a larger module or unit.

[0026] The technical solutions of this application will be described in detail below with reference to the accompanying drawings of the embodiments. It should be noted that the described embodiments are only a part of this application, and not all embodiments. In the following description, the "some embodiments" mentioned are only a subset of all possible embodiments, which may be the same or different subsets, and different embodiments can be combined with each other without conflict.

[0027] Figure 1This is a flowchart illustrating an edge computing inference method provided in an embodiment of this application. For example, the edge computing inference method provided in this embodiment is applied to an edge computing terminal. The edge computing terminal can be a dedicated industrial automatic optical inspection (AOI) terminal device equipped with a System on Chip (SoC) architecture. The device is equipped with an Advanced RISC Machine (ARM) main control chip, pre-installed with a Preempt Real-Time (PREEMPT_RT) Linux operating system, and integrates a Neural Processing Unit (NPU) computing unit, hardware timers, and a General Purpose Input Output (GPIO) signal acquisition module. This terminal can be deployed at an industrial production line inspection station and can independently complete localized tasks such as image acquisition, signal capture, and AI inference calculations without relying on cloud devices for data processing and command issuance. Its internal operating system performs time-sharing scheduling for various tasks such as camera image acquisition, NPU inference, and network communication, resulting in inherent scheduling jitter. In the exemplary application scenario, this terminal can connect to two Gigabit Ethernet cameras. Ethernet (GigE), adapted to the high-frequency trigger detection requirements of production lines at 30 times per second, used for visual inspection of appearance defects in industrial sheet metal, circuit boards, and other workpieces; see also Figure 1 The edge computing inference method provided in this application embodiment may include the following steps 101 to 105:

[0028] Step 101: Obtain the first moment and trigger identifier corresponding to the external trigger signal.

[0029] The external trigger signal is independent of the internal task scheduling clock of the edge computing terminal.

[0030] In some examples, the external trigger signal is a pulse-type trigger command actively generated by external hardware devices on the industrial production line. It is used to synchronously trigger the image acquisition action of the industrial camera. The generation and transmission timing of this signal are completely independent of the internal task scheduling clock of the edge computing terminal and are not affected by the scheduling status of various computing tasks within the terminal. The main body generating this signal is an external detection device such as a production line encoder or photoelectric sensor, which is the signal source that ensures the synchronization of production line image acquisition and workpiece conveying timing. It can be captured by hardware interrupts. The external sensor periodically generates level pulse signals according to the workpiece conveying progress and directly connects to the general input / output port of the edge computing terminal through hardware lines. The terminal captures the pulse signals in real time through the hardware interrupt service routine to complete the effective acquisition of the external trigger signal. For example, the incremental encoder matched with the spindle of the industrial production line will follow the workpiece conveying at a constant speed and output pulse-type external trigger signals at a fixed frequency of 30 times per second, providing a precise external trigger reference for each workpiece image acquisition.

[0031] The first moment is a hardware timestamp used to accurately record the actual occurrence time of an externally triggered event. It serves as the benchmark data for determining the trigger timing and calibrating the image acquisition timing. It is generated by a high-precision hardware timer built into the edge computing terminal chip, with a timing accuracy of up to one microsecond. It is completely independent of the operating system software scheduling logic and can avoid timing deviations caused by system multi-task switching. For example, when the rising edge of the pulse output by the encoder triggers a hardware interrupt in the terminal, the system immediately reads the real-time value of the chip's hardware timer. This precise hardware timing data is the first moment corresponding to this trigger, ensuring the authenticity and high accuracy of the trigger time record.

[0032] A trigger identifier is a unique sequence identifier assigned to each valid external trigger event. It is used to distinguish between multiple trigger events arriving in rapid succession, establish a unique binding relationship between external trigger signals and subsequent inference tasks, and prevent matching errors in multi-trigger, multi-frame image scenarios. After the edge computing terminal system is initialized, a global counter register is started. Each time a valid external trigger signal is successfully captured, the value of the counter register is automatically incremented sequentially, and the unique incremented sequence number is used as the trigger identifier corresponding to this trigger. For example, after the device is powered on and initialized, the counter is reset to zero. After the first external trigger signal is captured, a trigger identifier with sequence number 1 is generated. After the second trigger signal is captured, a trigger identifier with sequence number 2 is generated, and so on, incrementing sequentially to ensure that the identifier of each trigger record is unique and non-repeating throughout the process.

[0033] For example, the signal output terminal of the incremental encoder on the production line can be directly connected to the general input / output port hardware pin of the edge computing terminal through a dedicated shielded cable, ensuring that the pulse signal generated by the encoder can directly trigger the terminal hardware interrupt. When the industrial production line is running normally, the workpiece is conveyed at a uniform speed, and the encoder continuously outputs an independent external trigger signal at a fixed frequency. The timing of the signal generation is determined only by the mechanical operating state of the production line and has no relation to the working state of the internal neural network processor inference, camera frame buffer read / write, system background task scheduling, etc. of the edge computing terminal. Whenever the hardware interrupt service routine captures a valid trigger pulse, the terminal immediately reads the precise count value of the current hardware timer as the first moment corresponding to this trigger, and at the same time calls the global count register to complete the value increment, generating a globally unique trigger identifier and associating it with the current first moment for storage. This fully realizes the signal acquisition, timing recording, and identifier binding functions of this step, providing a precise timing and data binding foundation for subsequent image frame matching and deterministic inference tasks.

[0034] By implementing step 101, the edge computing terminal can accurately record the time information of external events, while avoiding the impact of the internal task scheduling process on the recording of the trigger time, thereby providing an accurate time reference for establishing the correspondence between external trigger events and inference tasks.

[0035] Step 102: Based on the trigger identifier, obtain the corresponding target frame image and the second moment of the target frame image.

[0036] In some examples, the target frame image is the original detection image acquired in real time by an industrial camera during the workpiece inspection process on the production line and stored in the equipment's cache space. It serves as the data carrier for subsequent edge-end inference and decision-making. Under multi-trigger intensive acquisition conditions, the camera continuously acquires and caches images, generating a large amount of continuous frame image data. The target frame image is the effective image frame with the highest temporal matching degree to a single external trigger event and can be used to correspond to the current trigger detection task. The edge computing terminal can retrieve the complete frame cache queue of the image acquisition device based on the unique trigger identifier it has acquired, compare the acquisition time information of each image frame in the cache one by one, and select the image frame with the smallest temporal deviation corresponding to the first moment of the trigger signal. This image frame is then determined as the target frame image corresponding to the current trigger identifier. For example, two gigabit Ethernet cameras continuously acquire images of industrial workpieces and cache frame data. For external trigger signals with a frequency of 30 times per second, the terminal searches the cache queue one by one and selects the optimal matching image corresponding to each trigger moment as the target frame image, ensuring the spatiotemporal correspondence between the image and the trigger event.

[0037] The second moment is a precise time stamp indicating when the target frame image is acquired by the hardware and captured by the camera. It represents the actual acquisition time of the image and serves as a benchmark parameter for subsequent calculations of timing deviations and verification of frame data validity. The second moment can be directly read from the hardware acquisition timestamp. Industrial cameras, based on the Video for Linux 2 (V4L2) driver framework, acquire images. When each frame is captured, the device hardware synchronously records a high-precision acquisition timestamp. After the terminal selects and determines the target frame image, it directly retrieves the original hardware acquisition timestamp of that frame as the second moment. The entire process is based on hardware timing, without software scheduling delay interference, and the timing accuracy can match the hardware microsecond-level timing standard. For example, the camera synchronously solidifies the frame acquisition timestamp the moment it completes the workpiece image acquisition. After the terminal matches and obtains the target frame image, it directly reads this original hardware timestamp as the second moment, providing accurate data support for subsequent timing deviation verification.

[0038] For example, an edge computing terminal equipped with a real-time preemptive kernel Linux system runs continuously. The terminal connects to a gigabit Ethernet camera through the video4Linux2 driver framework. During the camera's operation, it continuously acquires images of the workpiece and stores all image frames in the local frame buffer queue of the device according to the acquisition order. Each image frame has its own independent hardware acquisition timestamp. After obtaining the unique trigger identifier and first moment corresponding to a single detection task, the terminal uses the trigger identifier as a search index to traverse all unmatched image frame data in the current frame buffer queue. The terminal compares the time difference between the hardware acquisition timestamp of each image frame and the first moment of the trigger signal, and selects the image frame with the smallest time difference as the target frame image corresponding to this trigger identifier, ensuring the timing consistency between the external trigger action and the image acquisition action. After determining the target frame image, the terminal directly extracts the original acquisition timestamp that has been fixed in the hardware acquisition stage of the image frame and determines it as the second moment corresponding to this detection task, completing the binding of the trigger identifier, the target frame image, and the second moment.

[0039] By implementing step 102, external triggering events can be associated with the specifically acquired image data, enabling the edge computing terminal to determine the source of the target frame image involved in inference processing and its acquisition time, thereby improving the matching accuracy between image data and external triggering events.

[0040] Step 103: If the time difference between the first moment and the second moment is less than or equal to a preset deviation threshold, then perform the first inference processing on the target frame image based on the first inference path.

[0041] In some examples, the preset deviation threshold is a pre-configured, fixed maximum allowable timing deviation threshold within the edge computing terminal system. It is used to determine the validity of the timing match between external trigger events and camera image acquisition actions, and serves as a criterion for distinguishing between reliable and unreliable image frames. It can be pre-calibrated and configured based on the device hardware precision, production line trigger frequency, and measured jitter data. The parameters are fixed during the device initialization phase and can be stably invoked during operation. For example, combined with the hardware timing precision of a terminal with an Advanced Reduced Instruction Set Machine Architecture (ARPA) and a production line trigger frequency of 30 times per second, the preset deviation threshold is calibrated to 500 microseconds. This can effectively cover the controllable timing jitter range during normal device operation, while accurately identifying abnormal jitter conditions.

[0042] The first inference path is a standard high-precision inference computation link activated by the edge computing terminal for image frames that pass time-series verification. It is the default defect detection inference process of the device, which runs on the computing power resources of the terminal's dedicated neural network processor. It has the characteristics of high detection accuracy and comprehensive defect recognition dimensions, and is suitable for most routine detection scenarios with normal time-series matching. For example, the first inference path is a complete deep learning defect detection model inference link deployed on the neural network processor, which can accurately identify various defects such as minor scratches, stains, and deformations on the surface of the workpiece, meeting the high-precision industrial inspection standards.

[0043] The first inference processing is the complete artificial intelligence inference operation process performed by the terminal on the time-qualified target frame image through the first inference path. It includes the entire process of image preprocessing, model feature extraction, defect classification and decision, confidence calculation and other operations. For example, for a workpiece image with a time deviation of no more than 500 microseconds, the terminal starts a complete deep learning model operation to perform a comprehensive defect feature analysis on the target frame image and output high-precision detection results to ensure the accuracy and reliability of routine inspections on the production line.

[0044] For example, during the device initialization phase of the edge computing terminal, the parameters of the preset deviation threshold of 500 microseconds have been solidified, and the deployment of the complete deep learning detection model and the configuration of the neural network processor computing power scheduling have been completed. After the terminal acquires the first time, the second time and the matched target frame image corresponding to a single detection task, it automatically calculates the absolute time difference between the two hardware timestamps. The terminal compares the real-time calculated time difference with the preset deviation threshold. If the real-time time difference is less than or equal to 500 microseconds, it is determined that the current target frame image is a reliable acquisition frame with normal time matching, and there is no image misalignment problem caused by system scheduling jitter. At this time, the system automatically activates the preset first inference path, schedules the conventional computing power resources of the neural network processor, and performs complete first inference processing on the current target frame image. It sequentially completes a series of high-precision inference operations such as image standardization preprocessing, multi-layer network feature extraction, defect type determination and confidence output, and completes the detection and judgment entirely based on the standard AI model.

[0045] By implementing step 103, the temporal validity of the target frame image can be constrained, avoiding images with excessive time deviation from participating in inference processing. This reduces the problem of inaccurate inference results caused by the mismatch between external triggering events and image acquisition time, and improves the reliability of inference results.

[0046] Step 104: If the estimated total time of the first inference process is greater than the preset inference time limit during the first inference process, the first inference process is terminated, and the target frame image is subjected to second inference process based on the second inference path to obtain the inference result; otherwise, the first inference process is continued to obtain the inference result.

[0047] The computational resources required for the second inference process are lower than those for the first inference process.

[0048] In some examples, the estimated total time is the total time required to complete all inference operations for the entire frame of an image, calculated in real time by the edge computing terminal during the first inference process. This is used to determine in advance whether there is a risk of timeout for the current inference task and is a predictive parameter for achieving controllable inference latency. For example, during the first inference process of the neural network processor, the edge computing terminal counts the executed time and the remaining amount of computation in real time and dynamically calculates the estimated total time for the current frame to determine whether it exceeds the boundary of the device's real-time processing capability.

[0049] The preset inference time limit is the maximum allowable inference latency budget pre-configured and fixed in the edge computing terminal system. It is the time threshold for the complete inference output result of a single frame image, used to uniformly constrain the maximum latency of all inference tasks and eliminate long-tail latency and task backlog. The parameter fixed configuration can be completed during the equipment initialization stage according to the cycle time of the industrial automatic optical inspection production line, the camera trigger frequency, and the computing power performance of the equipment. For example, combined with the production line trigger frequency of 30 times per second and the computing power performance of the Advanced Reduced Instruction Set Machine Architecture on-chip system, the preset inference time limit is configured as 15 milliseconds as the latency constraint standard for a single frame inference task.

[0050] The second inference path is a lightweight backup detection inference link configured on the edge computing terminal. It is a fallback execution solution in the event of a timeout in the main inference link. The overall architecture does not rely on large deep learning models and adopts a traditional machine vision algorithm architecture to achieve the detection function. The second inference path has lower overall computing power and constant computation time, and can stably output detection results within a limited latency. For example, the second inference path is a traditional fast defect detection algorithm link based on grayscale histograms. Compared with deep learning model inference, the computation logic is more streamlined and the resource consumption is significantly lower than that of the first inference path.

[0051] The second inference process is a lightweight and rapid detection operation performed on the target frame image based on the second inference path after the terminal triggers the degradation mechanism. It belongs to the conservative and rapid decision process and is used to quickly output effective detection results in emergency situations where the main inference task is about to time out, ensuring that the production line detection is not interrupted. The overall computing resource consumption of the second inference process is significantly lower than that of the first inference process, and the processing time is stable and controllable, which can strictly adapt to the real-time requirements of the production line. For example, the second inference process realizes rapid screening of workpiece defects through lightweight operations such as grayscale feature statistics and pixel distribution discrimination, and can complete the detection and decision of the entire frame image in a very short time.

[0052] The inference result is the final workpiece inspection judgment conclusion output by the terminal through the first or second inference processing. It is used to characterize the workpiece quality status corresponding to the current target frame image and is the final effective output data of the visual inspection task. The inference result can be generated by the inference algorithm. Time-series qualified images are given priority to generate fine detection results through high-precision AI inference. Under time-out conditions, lightweight algorithms are used to generate fast downgraded detection results. Both types of results can be used as the basis for workpiece judgment on the production line. For example, the inference result includes basic judgment conclusions such as workpiece qualification, workpiece local defects, and workpiece serious defects, which can fully adapt to the quality judgment needs of industrial production lines.

[0053] For example, this solution employs an industrial automated optical inspection edge computing terminal equipped with an advanced RISC machine architecture system-on-a-chip and a pre-installed real-time preemptive kernel Linux system. During the device initialization phase, a preset inference time limit of 15 milliseconds is fixed, and a dual-path deployment of a high-precision deep learning inference model and a lightweight traditional visual inspection algorithm is completed simultaneously. During the execution of the first inference processing on a qualified target frame image, the terminal continuously calculates the estimated total time of the current inference task in real time and dynamically evaluates the overall task completion time. When the calculated estimated total time exceeds the preset inference time limit of 15 milliseconds, the terminal immediately stops the ongoing first inference processing, stops the computationally intensive deep learning model operation, and automatically switches to the second inference path. The second inference processing is performed on the same target frame image using a lightweight, low-resource-consumption traditional visual algorithm to quickly complete image feature analysis and defect judgment, and finally outputs a stable and effective inference result. If the real-time calculated estimated total time is less than or equal to the preset inference time limit, the terminal continues to execute the first inference processing normally, completes the full inference operation based on the high-precision AI model, and outputs the corresponding inference result.

[0054] By implementing step 104, the problem of the inference task not being completed in time due to the continuous occupation of computing resources by the first inference process can be avoided. At the same time, by dynamically selecting different inference paths according to the inference time status, the edge computing terminal can continue to output inference results while meeting the latency requirements, thereby improving the time controllability of the inference process. The entire execution logic can achieve rigid controllability of inference latency without interrupting the production line inspection process, effectively solving the problems of inference task accumulation and uncontrollable latency jitter in dense triggering scenarios, and continuously ensuring the stable output of industrial vision inspection tasks.

[0055] Step 105: Output the inference data packet, which includes the trigger identifier, the first time step, the second time step, and the inference result.

[0056] In some examples, the inference data packet is a structured and standardized data message generated by the edge computing terminal after completing a single image inference task. It is used to completely record the associated information of a single detection task, realize the one-to-one binding of external triggering events, image acquisition timing and inference decision results. It is the basic data carrier for industrial production line quality traceability, detection data statistics and abnormal working condition review. The composition of the inference data packet includes four core basic fields: trigger identifier, first moment, second moment and inference result. The field data all come from the acquisition, matching and inference process of the same detection task. The data source is unique and corresponds to each other.

[0057] For example, after completing the entire process of timing verification, frame image matching, and inference operation, the edge computing terminal reads all the valid basic data cached and stored during the lifecycle of this detection task. The terminal then structurally integrates and encapsulates the trigger identifier used to uniquely distinguish this trigger event, the first moment representing the actual external trigger time, the second moment representing the actual camera image acquisition time, and the inference result finally obtained after the first or second inference processing, generating an independent and complete inference data package and completing the output.

[0058] By implementing step 105, the correlation information between external triggering events, image acquisition process and inference results can be saved, effectively solidifying the binding relationship between triggering events, image acquisition and inference tasks. This provides reliable data support for industrial production lines to achieve traceability of detection results, abnormal jitter analysis and detection quality control, enabling subsequent tracking and analysis of the inference process based on the inference data package, thereby improving the integrity and traceability of inference data.

[0059] In summary, this embodiment of the application obtains the first moment and trigger identifier corresponding to an external trigger signal that is independent of the internal task scheduling clock of the edge computing terminal, and obtains the corresponding target frame image and second moment based on the trigger identifier. This establishes a correspondence between external trigger events and image data. Since the external trigger signal is not affected by the internal task scheduling of the edge computing terminal, it avoids the problem of uncertainty in the correspondence between trigger events and image acquisition caused by factors such as operating system task scheduling and computing resource contention, thereby improving the time consistency between inference task input data and actual trigger events. By comparing the time difference between the first moment and the second moment, and only performing inference processing based on the first inference path when the time difference meets a preset deviation threshold, the temporal validity of the target frame image can be judged. This allows for the selection of target frame images that match the time of the external trigger event, reducing the problem of inaccurate inference results caused by excessive deviation between the trigger signal and the image acquisition time, and improving edge computing performance. The reliability of edge computing terminals executing inference tasks is improved. During the first inference process, by determining whether the estimated total time of the first inference process exceeds the preset inference time limit, and terminating the first inference process if it does, the process switches to a second inference path with lower computational resource consumption. This avoids the problem of complex inference tasks continuously consuming computational resources and thus failing to complete in a timely manner. Since the computational resource consumption of the second inference path is lower than that of the first inference path, inference results can continue to be output even when inference time is limited, thereby improving the time controllability of the inference process. By outputting an inference data packet containing trigger identifiers, a first moment, a second moment, and the inference result, the correlation information between external trigger events, the image acquisition process, and the inference result can be preserved. This allows for subsequent tracing and analysis of the inference process based on the inference data packet. While ensuring the correspondence between the inference task and the external trigger event, it also improves the data integrity and traceability of the inference result. In summary, the edge computing inference method provided in this application improves the time determinism, result reliability, and data traceability of the edge computing inference process by establishing a deterministic association between external trigger events and target frame images, combined with timing verification, dynamic switching of inference paths, and an inference data traceability mechanism.

[0060] In some embodiments, the aforementioned step 101 may include: obtaining the system timer count value when the trigger event occurs through a hardware interrupt service routine, determining the system timer count value as the first moment; and generating a globally unique sequence number as a trigger identifier based on the receiving order of external trigger signals and according to a preset incrementing rule.

[0061] In some examples, the hardware interrupt service routine is a dedicated subroutine embedded in the underlying driver of the edge computing terminal and independent of the time-sharing task of the operating system. It is only momentarily awakened and runs when a valid level pulse is captured at the general-purpose input / output port. During its operation, it will not be preempted by background services such as camera acquisition and neural network processor operation. It is dedicated to handling the momentary acquisition of external hardware signals. For example, by connecting the output signal line of the incremental encoder to the pin of the general-purpose input / output port and configuring the rising edge of the pulse as the interrupt trigger condition, the hardware interrupt service routine starts running the moment the pulse arrives, without waiting for the task scheduling queue of the operating system.

[0062] A trigger event is a single, complete signal triggering behavior where an external hardware outputs a valid pulse level and is captured by the universal input / output port of an edge computing terminal. It is the prerequisite behavior for generating the first moment and trigger identifier. Each valid pulse corresponds to an independent trigger event. It can be generated by an external encoder or photoelectric sensor periodically outputting high and low level pulses as the workpiece is conveyed. When the level transition reaches the universal input / output port, a trigger event is determined to have occurred. For example, the spindle encoder of the production line moves at a constant speed with the sheet metal and outputs a pulse once for each fixed length of workpiece conveyed. Each change in the pulse level generates an independent trigger event, and the trigger frequency is stable at 30 times per second.

[0063] The system timer count value is the raw timing value continuously accumulated and output by the hardware timer built into the edge computing terminal chip. The hardware timer runs independently, with a timing accuracy of up to one microsecond. It is not affected by operating system software delays or fluctuations in computing load, and is the raw timing data that represents the actual occurrence time of the trigger event. For example, when the rising edge of the trigger pulse wakes up the hardware interrupt service routine, the program instantly reads the real-time count value of the hardware timer, which is 23641012 microseconds. This raw value is directly used as the first moment corresponding to this trigger event.

[0064] The preset increment rule is the counting update logic pre-set in the internal counting register of the edge computing terminal. It stipulates that for each complete and valid trigger event captured, the value stored in the register increases by a fixed unit value of 1. Only positive accumulation is performed throughout the process, and there is no value rollback, jump, or repeated assignment. For example, the counting register executes the increment rule of 1 for each trigger by default, and there will be no random increase, decrease, or reset operations, ensuring the stability and uniformity of the counting logic.

[0065] A globally unique sequence number is a sequence of numbers assigned sequentially to each trigger event according to a preset incrementing rule. This sequence number serves as a trigger identifier, binding the corresponding trigger event, target frame image, and inference task. There are no duplicates or overlaps in the sequence numbers corresponding to all trigger events, covering the entire operating cycle of the device. For example, after the device is powered on, the initial value of the register is zero. After the first trigger event is captured, the value is updated to 1, and after the second trigger event is captured, the value is updated to 2, and so on, generating mutually unique globally unique sequence numbers.

[0066] For example, during the device initialization phase of the edge computing terminal, all underlying driver configurations are completed. The signal output terminal of the incremental encoder on the production line is connected to the hardware pin of the general-purpose input / output port, the corresponding hardware interrupt service routine is attached, and the pulse rising edge trigger mode is set. At the same time, the global counter register is initialized, and the preset increment rule of incrementing the value by one for each trigger is fixed. When the production line is running normally, the encoder continuously outputs a pulse level signal following the workpiece conveyor. Each time the level transition reaches the general-purpose input / output port, a trigger event is generated, instantly waking up the hardware interrupt service routine. The hardware interrupt service routine is not affected by the inference of the terminal's internal neural network processor or the read / write of the gigabit Ethernet camera image buffer. To avoid task scheduling interference, the system directly reads the real-time output of the system timer count value from the chip's built-in hardware timer and determines this raw count value as the first moment corresponding to the current trigger event. Simultaneously, the counting logic is executed, and the global count register increments its value according to a preset increment rule to generate a globally unique sequence number specific to the current trigger event. This sequence number is then set as the trigger identifier corresponding to the external trigger signal, completing high-precision timing acquisition and unique identifier generation for a single trigger event. The entire process relies on the underlying hardware for execution, avoiding timing distortion and identifier duplication caused by system scheduling jitter, and providing accurate and reliable basic data for subsequent image frame matching and timing verification.

[0067] By implementing the above embodiments, the system timer count value at the time of the trigger event is obtained by using the hardware interrupt service routine as the first moment. Time information can be recorded directly when the external trigger signal arrives, reducing the impact of software task scheduling delay on the accuracy of trigger time recording and improving the accuracy of trigger event time acquisition. Furthermore, a globally unique sequence number is generated as a trigger identifier based on the receiving order of the external trigger signal. Different trigger events can be uniquely identified, avoiding confusion when multiple trigger signals arrive consecutively, and providing a clear data index basis for the subsequent target frame image matching process.

[0068] In some embodiments, step 102 may include: retrieving a target frame image from the frame buffer queue of the image acquisition device with the smallest time difference between the frame capture timestamp and the first moment; and determining the frame capture timestamp of the target frame image as the second moment.

[0069] In some examples, the image acquisition device is an industrial camera deployed at the workstation of an industrial automated optical inspection production line to acquire images of the workpiece surface in real time. It is responsible for continuously acquiring images according to the production line's triggering cycle, providing raw image data for edge inference. The image acquisition device uses a gigabit Ethernet camera, which has the characteristics of high-speed transmission, high frame rate, and high-precision hardware timing, and can adapt to high-frequency external triggering acquisition conditions. For example, two gigabit Ethernet cameras are deployed at the industrial inspection workstation, synchronously facing the workpiece conveying direction, and continuously acquiring inspection images according to the external triggering cycle of 30 times per second.

[0070] The frame buffer queue is a circular data storage queue opened in the memory of the edge computing terminal by the image acquisition device driver. It is used to cache multiple frames of image data acquired in real time by the camera and the corresponding temporal metadata. It can avoid the problem of image data loss and missed acquisition in high-speed image acquisition scenarios and provide a complete frame data source for temporal matching retrieval. For example, the frame buffer queue continuously caches consecutive image frames output by two gigabit Ethernet cameras. The queue also retains multiple frames of original images that have not been reasoned in time, which can be used by the terminal to complete temporal traversal and optimal frame retrieval.

[0071] A frame capture timestamp is a high-precision time stamp generated by hardware at the moment when an image acquisition device completes the photoelectric signal acquisition and imaging storage of a single frame image. It is used to uniquely record the true acquisition time of each frame image. The time base is consistent with the terminal hardware timer, with microsecond-level timing accuracy, and is not affected by the operating system software scheduling. For example, each workpiece inspection image stored in the frame buffer queue is synchronously bound with a microsecond-level frame capture timestamp, providing a precise timing basis for subsequent timing difference calculation and frame matching and screening.

[0072] For example, the industrial automated optical inspection edge computing terminal initializes two gigabit Ethernet cameras and configures the frame buffer queue through the video4Linux2 driver. During normal operation, the cameras continuously acquire workpiece images and store them in the memory frame buffer queue in the acquisition order. All image frames are synchronously bound to independent frame capture timestamps. After obtaining the trigger identifier and first moment corresponding to a single inspection task, the terminal uses the first moment as the time reference to traverse the frame capture timestamps of all valid image frames in the current frame buffer queue, calculates the absolute time difference between each frame capture timestamp and the first moment, and retrieves and filters the image frame with the smallest time difference. This image frame is then identified as the target frame image corresponding to the current trigger identifier. After identifying the target frame image, the terminal directly extracts the frame capture timestamp originally bound to the image frame and identifies this timestamp as the second moment corresponding to this inspection task, thus completing the precise time sequence binding of the trigger identifier, the target frame image, and the second moment.

[0073] By implementing the above embodiments, the target frame image with the smallest time difference between the frame capture timestamp and the first moment can be retrieved from the frame buffer queue of the image acquisition device. The image with the highest correlation to the time of the external triggering event can be selected as the inference input from multiple buffer frames, reducing the frame matching error caused by image acquisition delay or buffer queuing. At the same time, by determining the frame capture timestamp of the target frame image as the second moment, the actual image acquisition time can be obtained, providing an accurate basis for subsequent time deviation calculation, thereby improving the synchronization accuracy between the visual detection inference task and the external triggering event.

[0074] In some embodiments, the aforementioned edge computing inference method may further include: if the time difference between the first time and the second time is greater than a preset deviation threshold, then acquiring the historical frame image corresponding to the previous trigger identifier adjacent to the trigger identifier; performing inter-frame difference processing on the target frame image and the historical frame image to obtain a difference image; determining whether the target frame image is a repeated trigger acquisition frame based on the distribution characteristics of non-zero pixels in the difference image; if it is determined to be a repeated trigger acquisition frame, then using the historical inference result corresponding to the historical frame image as the inference result of the target frame image, and generating an inference data packet containing a reuse identifier; if it is determined to be a non-repeated trigger acquisition frame, then generating a detection result containing an invalid state flag and a trigger identifier.

[0075] In some examples, the historical frame image corresponding to the previous trigger identifier adjacent to the trigger identifier is the complete image frame data that has completed acquisition and inference operations, corresponding to the global unique sequence number of the previous level of the current abnormal trigger identifier under the device's ordered counting generation rule. This image frame is the previous round of valid acquisition frame immediately adjacent to the current frame, and complete image data and inference result data have been retained. For example, when the current trigger identifier sequence number is 10 and the timing deviation exceeds the threshold, the terminal retrieves the acquired and inferred workpiece image corresponding to the previous trigger identifier with sequence number 9 as the historical frame image.

[0076] A differential image is a feature image generated by the terminal through a pixel-level comparison algorithm, which performs frame-by-frame difference calculations on a target frame image that exceeds a time-series threshold and adjacent historical frame images. It is used to intuitively reflect the area and magnitude of image changes between two frames, and serves as a feature carrier for determining whether a workpiece has undergone actual displacement and whether there is repeated acquisition. The terminal can read the pixel matrix of the target frame image and the historical frame image, calculate the grayscale difference pixel by pixel, retain the effective pixels in the pixel change area, and eliminate completely identical background and workpiece areas in the two frames, ultimately generating a differential image that retains only the difference features. For example, if the workpiece position does not change significantly between the two frames and the image heights overlap, the differential image will only contain a small number of noisy pixels; if the workpiece has shifted or a new defect has appeared, the differential image will show a large area of ​​continuous non-zero pixel regions.

[0077] Repeated trigger acquisition frames are highly overlapping image frames obtained by multiple trigger acquisitions within a short period of time when the production line is running at high speed and trigger signals are output densely. These frames do not contain new detection information and do not require repeated reasoning. Based on the distribution characteristics of non-zero pixels in the differential image, the process of determining whether the target frame image is a repeated trigger acquisition frame is a pixel feature statistical discrimination. After the terminal completes the generation of the differential image, it counts the number, distribution range, and connectivity features of non-zero pixels, and combines this with a fixed noise threshold for the industrial inspection scenario to complete the determination. When the number of non-zero pixels in the differential image is small, the distribution is scattered, and it is only a random noise distribution, it is determined that there is no substantial change in the image before and after the frame, and the current target frame image is a repeated trigger acquisition frame. When the number of non-zero pixels in the differential image is large, the distribution is continuous, and it exceeds the noise threshold, it is determined that the workpiece state has changed in a real way, and the current target frame image is a non-repeated trigger acquisition frame. For example, high-frequency triggering on the production line causes two consecutive image acquisitions to be basically the same, and the differential image only contains sporadic noise pixels. Based on this, the terminal determines that the current frame is a repeated trigger acquisition frame.

[0078] Historical inference results are valid detection conclusions that have been calculated and output in the previous trigger frame. The reuse flag is a dedicated field used to mark that the current frame has not undergone re-inference and directly uses the historical results. The invalid state flag is a dedicated field used to mark that the current frame has a timing anomaly and that the image has undergone real changes, making it unusable for valid detection. The terminal can automatically match the corresponding output strategy based on the judgment results of the inter-frame difference features. For duplicate frames, existing inference data is directly reused and a reuse flag is added; for frames with abnormal changes, an invalid state is directly marked and the trigger information is retained. For example, when the current frame is determined to be a duplicate trigger acquisition frame, the terminal uses the historical inference result of the previous frame's workpiece qualification and generates a complete inference data packet carrying the result reuse flag. When it is determined to be a non-duplicate trigger acquisition frame, a detection result carrying the invalid state flag and the current trigger flag is generated for subsequent anomaly tracing.

[0079] For example, during operation, the industrial automated optical inspection edge computing terminal continuously caches each acquired image frame and its corresponding inference result data in sequence according to the trigger identifier. When the terminal calculates that the time difference between the first and second moments of the current inspection task is greater than a preset deviation threshold of 500 microseconds, it immediately retrieves the previous trigger identifier immediately adjacent to the current trigger identifier and retrieves the corresponding historical frame image and historical inference data. The terminal performs pixel-by-pixel inter-frame differential processing on the target frame image that exceeds the time threshold and the cached historical frame image, removes overlapping areas of the image, and generates a differential image to characterize the differences in the image. The terminal further statistically analyzes the number and spatial distribution characteristics of non-zero pixels within the differential image to distinguish those caused by random noise. The system classifies and determines whether a frame is a repeating or non-repeating trigger acquisition frame based on subtle differences and significant differences caused by actual changes in the workpiece. If the current target frame image is determined to be a repeating trigger acquisition frame, the terminal does not initiate a new inference operation but directly reuses the historical inference results corresponding to the historical frame image as the detection conclusion for the current frame. At the same time, it encapsulates and generates a standardized inference data package carrying a result reuse identifier, a trigger identifier, and timing information. If the current target frame image is determined to be a non-repeating trigger acquisition frame, it indicates that the timing deviation exceeds the standard and there are real and valid changes in the image. The frame data cannot be used for reliable detection, and the terminal immediately generates a detection result carrying an invalid status marker and a current trigger identifier, completing the closed-loop output of this round of anomaly detection task.

[0080] Through the implementation of the above embodiments, when the time difference between the first and second moments exceeds a preset deviation threshold, the historical frame image corresponding to the adjacent previous trigger identifier is further acquired. Inter-frame differential processing is used to determine whether the target frame image is a repeatedly triggered acquisition frame. This distinguishes between repeated images caused by acquisition anomalies and valid images that have actually changed, avoiding unnecessary inference calculations due to repeated trigger acquisition. When a repeatedly triggered acquisition frame is determined, historical inference results are reused to reduce computational resource consumption caused by repeated inference. When a non-repeatedly triggered acquisition frame is determined, generating detection results containing invalid state markers prevents unreliable images from entering subsequent detection processes, improving the accuracy of detection results. Without increasing computational overhead, various abnormal working conditions in densely triggered scenarios can be accurately distinguished, avoiding both inference queue accumulation and latency loss caused by invalid repeated inference, and preventing missed detections caused by directly discarding real abnormal images. This effectively improves the fault tolerance and reliability of edge-end industrial vision inspection.

[0081] In some embodiments, terminating the first inference process if the estimated total time of the first inference process exceeds a preset inference time limit may include: obtaining the number of inference layers currently completed by the preset artificial intelligence inference model and the total number of inference layers in the preset artificial intelligence inference model; determining the estimated remaining time to complete the remaining inference layers based on a first ratio of the number of completed inference layers to the total number of inference layers and the inference time consumed from the start of the first inference process to the current moment; obtaining the current operating temperature of the edge computing terminal and the current load rate of the neural network processor; correcting the estimated remaining time based on a first deviation between the current operating temperature and a preset reference temperature and a second deviation between the current load rate and a preset reference load rate to obtain a corrected estimated remaining time; and sending an inference termination instruction to the preset artificial intelligence inference model if the sum of the consumed inference time and the corrected estimated remaining time exceeds the preset inference time limit.

[0082] In some examples, the preset AI inference model is a complete deep learning inference model deployed inside the neural network processor of the edge computing terminal to perform high-precision defect detection. It is the computational model corresponding to the first inference path, and has the characteristics of detailed feature extraction and high detection accuracy. It is suitable for high-precision detection scenarios of conventional time-series qualified images. For example, the preset AI inference model is a multi-layer convolutional neural network model adapted to industrial workpiece defect detection. The network structure is fixed and the overall number of layers is constant. It is used as the main inference model for routine inspection on the production line.

[0083] The total number of inference layers is the total number of complete network layers fixed in the preset AI inference model. It is an inherent static parameter of the model, and its value remains constant during device operation. It is the benchmark parameter for calculating the inference progress ratio. For example, the total number of inference layers is fixed at 30 layers and does not change with device load or temperature. The number of inference layers that have been executed is the number of network layers that the preset AI inference model has completed at the current moment during a single first inference process. It is used to characterize the real-time execution progress of the model's inference. For example, the preset AI inference model contains a 30-layer network structure. When the inference execution reaches the middle stage, the device reads in real time that 18 inference layers have been executed. The first ratio is a dimensionless ratio of the number of inference layers that have been executed to the total number of inference layers. It is used to objectively reflect the overall completion progress of the current inference task and is a parameter for initially estimating the remaining inference time. For example, if 18 layers have been completed and the total number of layers is 30, the first ratio is calculated to be 0.6.

[0084] The consumed inference time is the cumulative runtime from the start of the first inference process to the current sampling time. It is used to characterize the computation time already occupied by the current inference task and is the basic timing parameter for timeout judgment. It can rely on the high-precision hardware timer of the edge computing terminal to start timing when the inference task starts and accumulate the consumed inference time in real time. For example, after the model inference starts, when the progress reaches 60%, the hardware timing statistics show that the consumed inference time is 8 milliseconds.

[0085] The estimated remaining time is a preliminary estimate by the terminal based on the model inference progress and the time already consumed, representing the theoretical time required to complete the inference of all remaining network layers. It does not take into account the impact of hardware operating condition fluctuations and is only used as an initial estimate of the remaining time. The terminal can combine the first ratio with the time already consumed to calculate the remaining computation time according to the linear progress relationship. For example, based on an inference progress of 0.6 and a time already consumed of 8 milliseconds, the estimated remaining time is approximately 5.3 milliseconds.

[0086] The current operating temperature is the surface temperature of the edge computing terminal's main control chip in real time. When the device operates under high load for a long time, the temperature will rise, which will cause the chip to reduce its frequency and computing power, resulting in increased inference time. The terminal's built-in temperature sensing module can periodically collect the chip's real-time temperature data for the system to call for calculation; for example, after the device has been running a detection task for a long time, the current operating temperature is collected in real time as 65 degrees Celsius.

[0087] The current load rate is the proportion of computing resources occupied by the neural network processor at the current moment. It is used to characterize the real-time computing power of the terminal. An excessively high load rate will cause a decrease in inference operation speed and an increase in single-frame time. The neural network processor can drive the real-time statistics of the current computing power occupied ratio and output the load value in the form of a percentage. For example, in the case of multi-frame task queuing operation, the current load rate of the neural network processor is 85%.

[0088] The preset reference temperature is the standard operating temperature under normal rated operating conditions of the equipment. It serves as a temperature reference parameter for determining whether the equipment exhibits abnormal temperature rise or computing power throttling. This temperature is fixed during the equipment commissioning phase. For example, the preset reference temperature is calibrated to 55 degrees Celsius. The first deviation is the difference between the current operating temperature and the preset reference temperature. It is used to quantify the degree of hardware temperature rise and is a temperature correction parameter for adjusting inference time. For example, the difference between the current operating temperature of 65 degrees Celsius and the preset reference temperature of 55 degrees Celsius yields a first deviation of 10 degrees Celsius.

[0089] The preset baseline load rate is the standard load ratio under the rated stable operating conditions of the neural network processor. It serves as a benchmark parameter for determining whether the computing power is overloaded and is used to measure whether the current computing power load is within the normal range; for example, the preset baseline load rate is calibrated to 60%. The second deviation is the difference between the current load rate and the preset baseline load rate. It is used to quantify the degree of computing power overload and is a computing power correction parameter for adjusting the remaining inference time; for example, the difference between the current load rate of 85% and the preset baseline load rate of 60% yields a second deviation of 25%.

[0090] The corrected estimated remaining time is the final remaining time obtained by dynamically compensating for the initial estimated remaining time by combining temperature deviation and computing load deviation. It can truly adapt to actual working conditions such as device temperature rise / fall and computing load preemption, and greatly improve the accuracy of timeout prediction. The terminal can set the corresponding correction coefficient according to the degree of deviation between the first deviation and the second deviation to amplify and compensate the theoretical estimated remaining time, so as to obtain the remaining time that fits the actual computing load state. For example, after dual correction of temperature and load, the initial estimated remaining time of 5.3 milliseconds is corrected to 7.2 milliseconds.

[0091] The total estimated time can be obtained by adding the consumed inference time to the corrected actual remaining time. The system strictly compares the estimated total time with the preset inference time limit. If the estimated total time exceeds the limit, the system immediately issues an instruction to stop the model's forward inference operation and promptly triggers the degradation processing mechanism of the second inference path to avoid unnecessary computing power occupation and excessive latency issues. For example, if the consumed inference time is 8 milliseconds and the estimated remaining time is 7.2 milliseconds after correction, the total time exceeds the preset inference time limit of 15 milliseconds. The system immediately issues an inference abort instruction to terminate the current artificial intelligence model inference task.

[0092] For example, during the device initialization phase of the edge computing terminal, the preset artificial intelligence inference model is solidified and deployed, and the parameters of the preset benchmark temperature and preset benchmark load rate are calibrated and stored. Simultaneously, a preset inference duration limit of 15 milliseconds is configured. During the execution of the first inference processing on a qualified target frame image, the terminal continuously reads the model's running status and device hardware operating parameters in real time. The terminal parses and determines that the total number of inference layers in the current preset artificial intelligence inference model is fixed at 30 layers. It then counts the number of inference layers that have been completed in real time, calculates the first ratio between the completed layers and the total number of layers, and simultaneously uses a hardware timer to count the inference time consumed in this inference task. Based on the aforementioned parameters, the terminal calculates the initial estimated remaining time and simultaneously reads the terminal's current... The current operating temperature and the current load rate of the neural network processor are compared with the preset reference temperature and preset reference load rate to obtain the corresponding first deviation and second deviation. The terminal combines the temperature deviation and load deviation to perform weighted compensation correction on the theoretically estimated remaining time, resulting in a corrected estimated remaining time that fits the real-time operating status of the device. The terminal adds the consumed inference time to the corrected estimated remaining time and compares the accumulated estimated total time with the preset inference time limit. When the overall estimated total time exceeds the time limit, the terminal immediately issues an inference abort command to the preset artificial intelligence inference model, forcibly terminating the current high-precision first inference processing flow, providing a prerequisite for switching to the low-computing-cost second inference path and completing the rapid downgrade decision.

[0093] By implementing the above embodiments, the number of inference layers completed by the artificial intelligence inference model, the total number of inference layers, and the inference time consumed can be obtained. Based on the current inference execution status, the time required to complete the remaining inference layers can be predicted, enabling the edge computing terminal to dynamically evaluate whether the first inference processing can meet the latency requirements. Furthermore, by combining the current operating temperature and the current load rate of the neural network processor to correct the estimated remaining time, the impact of changes in the edge computing terminal's operating status on the inference speed can be considered, improving the accuracy of inference time prediction. When the total time exceeds the preset inference time limit, an inference abort command is sent, which can release computing resources in advance and avoid the inference task timeout affecting the real-time execution of subsequent visual detection tasks. That is, through a multi-dimensional dynamic correction prediction mechanism, the problem of inaccurate prediction for fixed time is completely solved, achieving accurate and rigidly controllable inference latency under complex edge computing conditions.

[0094] In some embodiments, the aforementioned second inference path may include inference processing based on a preset lightweight detection algorithm. The preset lightweight detection algorithm is deployed in an independent hardware acceleration unit in the edge computing terminal. The independent hardware acceleration unit completes resource locking during the system initialization phase of the edge computing terminal so as not to be preempted by other tasks during operation.

[0095] In some examples, the preset lightweight detection algorithm is a traditional machine vision detection algorithm that differs from artificial intelligence deep learning models. It is the computational logic of the second inference path, used to perform a fast, low-computational-cost defect discrimination task when the main inference path is about to time out. This algorithm abandons complex network layer operations and relies on fixed pixel statistics and feature rules to complete the detection. The overall operation process is simple, the number of operation steps is fixed, and it has the characteristics of constant processing time, low resource consumption, and extremely low execution latency. For example, the preset lightweight detection algorithm uses a grayscale histogram feature detection algorithm. By statistically analyzing the grayscale pixel distribution of the target frame image, it can quickly determine whether there are common defects such as stains, blemishes, and abnormal brightness on the surface of the workpiece. It can complete the detection and judgment of a single frame image in a very short time.

[0096] An independent hardware acceleration unit is a dedicated hardware computing module within an edge computing terminal, independent of the neural network processor. It is specifically designed to accelerate the computation of a preset lightweight detection algorithm, providing dedicated hardware computing power support for the second inference path of this application. This hardware unit has independent computation registers and data cache resources, does not participate in the system's general task scheduling, and can achieve hardware-level acceleration of the detection algorithm. For example, the edge computing terminal's on-chip system has a built-in independent vision acceleration hardware unit. After the system initialization is completed and resources are locked, this hardware unit is used only to run the preset lightweight detection algorithm throughout the entire process and does not participate in other system tasks such as image caching, log computation, and network transmission acceleration.

[0097] For example, during the device initialization phase, the edge computing terminal completes the full deployment and parameter calibration of a preset lightweight detection algorithm, and solidifies the grayscale histogram fast defect detection algorithm as the dedicated execution algorithm for the second inference path. At the same time, the terminal performs resource locking configuration on the on-chip integrated independent hardware acceleration unit. Through kernel resource isolation configuration, all computing resources and cache resources of the hardware unit are locked, restricting the resource access permissions of all other services such as camera image acquisition tasks, neural network processor main inference tasks, and network communication tasks within the system, so that the independent hardware acceleration unit maintains a state of exclusive resource access throughout the entire device operation cycle. When the first inference process is predicted to exceed the preset inference time limit and is terminated during operation, the system immediately schedules the preset lightweight detection algorithm to start running in the independent hardware acceleration unit, and completes the rapid feature analysis and defect judgment of the target frame image by relying on dedicated isolated hardware resources.

[0098] Through the implementation of the above embodiments, the second inference path is set as an inference processing based on a preset lightweight detection algorithm and deployed as an independent hardware acceleration unit in the edge computing terminal. It can utilize independent computing resources to perform degraded inference tasks, reducing the dependence of the second inference processing on the main computing resources. Furthermore, by completing the resource locking of the independent hardware acceleration unit during the system initialization phase, the acceleration unit is not preempted by other tasks during its operation, thereby ensuring that the second inference path has a stable execution time. Even when the first inference path cannot meet the latency requirements, it can still quickly output inference results, improving the real-time performance and stability of the edge vision detection system. Since there is no preemption or scheduling waiting of hardware resources throughout the process, the computation time of the preset lightweight detection algorithm remains constant, and the second inference processing can be completed stably within the preset inference time limit and output effective inference results. This effectively solves the technical problems of no stable degraded output and uncontrolled inference latency after the main inference times out, ensuring that high-frequency detection tasks on industrial production lines are not interrupted or fail.

[0099] In some embodiments, after the aforementioned output inference data packet, the edge computing inference method may further include: obtaining the time difference between a first moment and a second moment in multiple inference data packets within a preset statistical period; determining the trigger-frame synchronization jitter distribution within the preset statistical period based on the multiple time differences; and adjusting a preset deviation threshold and / or a preset inference duration upper limit based on the trigger-frame synchronization jitter distribution.

[0100] In some examples, the preset statistical period is a fixed time interval pre-configured by the edge computing terminal for batch collection of multiple sets of time-series deviation data. The terminal only performs jitter statistics and parameter adjustment uniformly after a single complete period, avoiding statistical distortion caused by single or small amounts of detection data and ensuring that the parameter adjustment has data representativeness. For example, combined with an external trigger frequency of 30 times per second, the preset statistical period is configured to 10 minutes. Within a single period, tens of thousands of inference data packets containing the time difference between the first and second moments can be collected to form sufficient data samples to support statistical analysis.

[0101] Trigger-frame synchronization jitter distribution is the distribution pattern of timing deviation values ​​formed by the terminal after statistically summarizing the timing differences extracted from all inference data packets within a single preset statistical period. It can intuitively reflect the concentrated interval, maximum offset range, and frequency of offset occurrence between the external trigger signal and the camera-acquired frames over a period of time. It serves as a statistical basis for evaluating the synchronization performance of the equipment and adjusting control parameters. For example, the statistics show that most timing differences are concentrated within 200 microseconds, a very small number of differences are close to 500 microseconds, and there are no timing deviations exceeding 500 microseconds, thus forming a concentrated trigger-frame synchronization jitter distribution with relatively small amplitude.

[0102] The process of adjusting the preset deviation threshold and / or preset inference duration upper limit based on the trigger-frame synchronization jitter distribution involves the terminal judging the current device synchronization performance margin based on the complete jitter distribution characteristics, and specifically correcting two types of parameters: the preset deviation threshold used for frame synchronization verification and the preset inference duration upper limit used for inference latency constraints, balancing timing matching accuracy and production line detection real-time performance. The terminal can extract characteristic indicators such as the mean, extreme values, and interval proportions of the jitter distribution, and automatically amplify or reduce parameters according to preset adjustment rules. It can adjust one type of parameter alone or simultaneously correct both types of parameters. For example, if the overall amplitude of the trigger-frame synchronization jitter distribution is too small, the terminal appropriately reduces the preset deviation threshold to increase the stringency of frame matching; if there is a small but large offset in the jitter distribution, the preset deviation threshold is appropriately relaxed, and the preset inference duration upper limit is slightly increased to reduce timing misjudgment and inference timeout degradation.

[0103] For example, during the device initialization phase, the edge computing terminal completes the configuration of a preset statistical period of 10 minutes, and initially calibrates a preset deviation threshold of 500 microseconds and a preset inference time limit of 15 milliseconds. After each inference data packet is output, the terminal automatically caches the time difference calculated between the first and second moments in the data packet into the periodic statistical data set, continuously accumulating the temporal deviation data of all detection tasks within a single preset statistical period. When the 10-minute preset statistical period is completed, the terminal stops adding new data, traverses all temporal difference samples in the cache to complete statistical calculations, divides different deviation intervals, and counts the number of samples in each interval. The system generates a complete trigger-frame synchronization jitter distribution. The terminal reads the maximum jitter amplitude, jitter concentration range, and other characteristic indicators contained in the jitter distribution, and performs adaptive parameter correction operations according to the built-in parameter adjustment rules. If the jitter distribution shows that most timing deviations are stable below 300 microseconds, the terminal synchronously lowers the preset deviation threshold to 350 microseconds, tightens the frame synchronization verification standard, and filters out matching errors caused by slight jitter. If the jitter distribution shows a large number of timing differences close to 500 microseconds, the terminal appropriately relaxes the preset deviation threshold and slightly increases the preset inference time limit to reduce the situation where a large number of images are marked as invalid frames and inference frequently triggers degradation paths.

[0104] By implementing the above embodiments, the time difference between the first moment and the second moment in multiple inference data packets within a preset period can be statistically analyzed to obtain the synchronization deviation between the trigger event and image acquisition during actual operation. Furthermore, by determining the trigger-frame synchronization jitter distribution based on multiple time differences, the stability of trigger synchronization during edge computing inference can be quantified. Based on this jitter distribution, the preset deviation threshold and / or the preset inference duration upper limit can be adjusted so that the inference parameters can adapt to changes in the actual operating environment, thereby improving the adaptability of the visual inspection system under different working conditions.

[0105] In some embodiments, the aforementioned dynamic adjustment of the preset deviation threshold and / or preset inference duration upper limit based on the trigger-frame synchronization jitter distribution may include: acquiring product yield data of the production line corresponding to the edge computing terminal; constructing a jitter-yield correlation model based on the trigger-frame synchronization jitter distribution and product yield data; predicting the expected yield under different inference duration configurations in the next statistical period according to the jitter-yield correlation model; and adjusting the preset inference duration upper limit to the target inference duration, wherein the target inference duration enables the expected yield to meet the preset yield constraint and maximizes the inference completion rate.

[0106] In some examples, the product yield data of the production line corresponding to the edge computing terminal is the proportion of the number of workpieces judged as qualified by visual inspection to the total number of inspected workpieces in a single statistical period. It is a production indicator that directly reflects the stability of the production line inspection and the level of product quality control. The product yield data can intuitively show the degree of impact of mechanisms such as timing jitter, inference timeout degradation, and invalid frame rejection on the final inspection results, providing a real production dimension for parameter optimization. For example, if the equipment completes 10,000 workpiece inspections in a single statistical period and the number of qualified workpieces is 9,920, the corresponding product yield data for the statistical period is 99.2%.

[0107] The jitter-yield correlation model is a quantitative relationship model built based on the trigger-frame synchronization jitter distribution characteristic parameters and production line product yield data. It is used to characterize the inherent mapping law between timing jitter amplitude, jitter distribution range, inference timeout frequency, timing abnormal conditions, and production line detection yield. This model can quantify the impact weight of different timing jitter states and different inference duration parameter configurations on the final product detection quality, breaking through the traditional single logic of relying solely on hardware condition parameter tuning, and realizing intelligent parameter tuning based on production quality feedback. For example, the model can fit the corresponding relationship that increased jitter amplitude and excessively short inference duration leading to frequent downgrade decisions will directly result in a decrease in the effective yield of the production line.

[0108] The expected yield under different inference duration configurations in the next statistical period is the theoretical yield value of the production line corresponding to different preset inference duration upper limit parameters under future operating conditions, predicted by the jitter-yield correlation model based on historical time series data and production data. This expected yield can predict the production quality effect after parameter adjustment in advance, avoiding problems such as decreased detection accuracy and increased false positive rate caused by blindly modifying the inference duration parameter. For example, the model predicts the expected yield corresponding to three inference duration configurations of 14 milliseconds, 15 milliseconds, and 16 milliseconds, respectively, and obtains differentiated production quality prediction results under different parameters.

[0109] The target inference duration is the optimal upper limit parameter for inference duration selected from multiple sets of candidate inference duration parameters. It is a configuration parameter that simultaneously meets production line quality constraints and equipment operating efficiency. The selection criteria for this parameter are based on dual constraints. First, it is necessary to ensure that the expected yield predicted by the model meets the minimum yield standard set by the production line, thus satisfying production quality constraints. Second, it is necessary to select the duration configuration that maximizes the inference completion rate of the neural network processor within the range of compliant parameters, achieving bidirectional optimization of detection quality and inference efficiency. All candidate inference duration configurations can be traversed, parameters with unacceptable yield can be eliminated, and the parameter with the highest inference completion rate can be selected as the target inference duration from the remaining compliant parameters. For example, within the range of parameters that meet the preset yield standard of the production line, the optimal duration that can achieve a 100% inference completion rate can be selected as the target inference duration.

[0110] For example, after completing the timing jitter statistics for a single preset statistical cycle and generating the current round's trigger-frame synchronization jitter distribution, the edge computing terminal retrieves the complete workpiece inspection logs and inference data packets within the current cycle, and statistically summarizes them to obtain the product yield data for the corresponding production line. The terminal uses the timing jitter distribution feature parameters and cycle yield data as training samples to complete the iterative fitting of the current round's data, updates and solidifies the jitter-yield correlation model, and accurately establishes the quantitative mapping relationship between timing jitter status, inference duration parameters, and product inspection yield. Through the iterated correlation model, the terminal batch-predicts the expected yield results corresponding to multiple sets of different inference duration configuration parameters in the next statistical cycle, and sorts out the different parameters. The system assesses the quality level of detection and the efficiency of inference completion. The terminal uses the production line's preset yield constraints as a basic threshold to filter all candidate parameters for inference duration that meet quality standards. It then further selects the parameter with the highest inference completion rate from the compliant parameter group as the target inference duration. Finally, the terminal dynamically adjusts the system's original preset inference duration upper limit to the selected target inference duration. This ensures that during the next cycle of operation, the equipment can guarantee a stable product detection yield that meets production quality control requirements, while minimizing the frequency of inference timeouts and downgrade decisions, thereby improving the overall inference task completion rate and achieving intelligent and dynamic optimization of edge computing inference latency parameters for production adaptability.

[0111] By implementing the above embodiments, production line product yield data is obtained, and a jitter-yield correlation model is constructed by combining the trigger-frame synchronization jitter distribution. This establishes a correlation between synchronization stability parameters and visual inspection results, enabling inference parameter adjustments to be optimized based on actual inspection results. Furthermore, by predicting the expected yield under different inference duration configurations and selecting the target inference duration that satisfies preset yield constraints and maximizes the inference completion rate, a balance can be achieved between inspection quality and inference efficiency. This avoids the problem of simply increasing inference latency leading to reduced processing efficiency, or decreasing inference latency leading to decreased inspection accuracy.

[0112] In some embodiments, the aforementioned output inference data packet may include: obtaining the model version identifier, confidence information, degradation indication information, and image acquisition device identifier corresponding to the output of the first inference process or the second inference process; generating a data packet digest signature based on the unique device identifier and current timestamp of the edge computing terminal; and combining the model version identifier, confidence information, degradation indication information, image acquisition device identifier, data packet digest signature and trigger identifier, first time moment, second time moment, and inference result into an inference data packet.

[0113] In some examples, the model version identifier is a unique identifier used to characterize the version of the artificial intelligence model called in the current inference operation. It is used to distinguish between multiple generations of detection models that are iteratively updated within the terminal and to clarify the baseline of the algorithm capability corresponding to the detection result of a single frame. For example, the high-precision defect detection model currently in effect on the terminal is configured with a fixed version number. When the system performs the first inference process, it automatically reads the corresponding number and writes it into the data field to distinguish the detection differences before and after the model iteration.

[0114] Confidence information is the probability score data output by the artificial intelligence model for the defect judgment result of the current target frame image. It is used to characterize the credibility of the inference judgment result and is an important auxiliary indicator for judging the validity and reference value of the detection result. In the deep learning model inference process of executing the first inference processing, the model outputs the confidence value of the corresponding category after completing the feature classification judgment. If the traditional visual rule algorithm used in the second inference processing does not have the ability to output probability, it will output a compliance confidence value by default.

[0115] The degradation indication information is a status field used to mark the type of the current inference task execution path. It is used to distinguish whether the current detection result comes from the high-precision first inference path or the lightweight second inference path, and can accurately identify whether inference timeout degradation processing has occurred. For example, if the entire deep learning inference process is completed normally, a normal inference mark is generated. If the main inference is stopped due to prediction timeout and the lightweight algorithm is switched, a degradation judgment mark is generated.

[0116] The image acquisition device identifier corresponding to the target frame image is a unique device number used to distinguish multiple image acquisition devices. It is used to accurately locate the physical acquisition source of the current target frame image and adapt to industrial inspection conditions where multiple cameras acquire images in parallel. For example, if two image acquisition devices are deployed at the workstation, each retains an independent device identifier, the terminal can distinguish which camera acquired the current frame through the identifier.

[0117] A data packet digest signature is an encrypted verification digest generated by an edge computing terminal based on the device's inherent identity and real-time time information. It is used to ensure the integrity and tamper-proof nature of the inference data packet content, and to achieve reliable evidence storage of the detection data. The terminal can read the unique device identifier embedded in the device and the system timestamp of the data packet generation time, and generate an irreversible unique signature string through digest hash operation. Each inference data packet corresponds to a unique digest signature. For example, detection data packets generated at different times have completely different digest signatures, which can effectively prevent data tampering, data replacement and forged record problems.

[0118] Device identity information, model version information, inference status information, quality reliability information, time-series binding information, and final judgment results can be uniformly and structurally integrated to form a standard output data package with complete fields, closed logic, and full-dimensional auditability.

[0119] For example, after completing the inference operation of a single frame target image and obtaining the inference result, the industrial automatic optical inspection edge computing terminal first reads the model version identifier corresponding to this inference task, synchronously collects the confidence information output by the model, and generates corresponding degradation indication information based on whether a path switch occurred during this inference. The terminal retrieves the camera number bound during the target frame image caching stage to obtain the corresponding image acquisition device identifier. The terminal further reads the unique device identifier fixed by the device at the factory and the current system's precise timestamp, and generates a unique corresponding data packet digest signature through local digest operation. The terminal uniformly organizes and combines the above-mentioned newly added fields with the basic fields. The basic fields include the trigger identifier used to uniquely bind the trigger event, the first moment representing the actual trigger time, the second moment representing the image acquisition time, and the final inference result generated this time. All fields are encapsulated into a complete, independent, auditable, and tamper-proof standardized inference data packet according to a preset organized format and output.

[0120] By implementing the above embodiments, adding model version identifiers, confidence information, degradation indication information, and image acquisition device identifiers to the inference data packets can completely record the model status, the reliability of the detection results, and the data source information during the visual inspection process, thereby improving the information integrity of the inference results. Furthermore, by generating a data packet digest signature based on the unique device identifier of the edge computing terminal and the current timestamp, the inference data packets can be identified and associated, reducing the problem of unclear information correspondence during data transmission or storage. By combining the above information with the trigger identifier, the first moment, the second moment, and the inference results to form an inference data packet, the entire visual inspection process can be tracked and analyzed, improving the verifiability and traceability of the edge computing inference results. In other words, through this structured encapsulation method, each industrial visual inspection task can form a complete data traceability chain, accurately distinguishing model version differences, camera acquisition sources, inference degradation conditions, and the reliability of results, significantly improving the standardization, reliability, and production line quality traceability capabilities of the edge computing visual inspection results.

[0121] Furthermore, as an implementation of the aforementioned method embodiments, this application also provides an edge computing inference device for implementing the aforementioned method embodiments. This device embodiment corresponds to the aforementioned method embodiments. For ease of reading, this edge computing inference device embodiment will not repeat the details of the aforementioned method embodiments one by one, but it should be understood that the device in this application embodiment can correspondingly implement all the contents of the aforementioned method embodiments. For example... Figure 2 As shown, the edge computing inference device 20 includes: a signal acquisition unit 201, an image acquisition unit 202, a time difference processing unit 203, an inference processing unit 204, and a data output unit 205. The signal acquisition unit 201 is used to acquire a first moment and a trigger identifier corresponding to an external trigger signal, wherein the external trigger signal is independent of the internal task scheduling clock of the edge computing terminal. The image acquisition unit 202 is used to acquire a corresponding target frame image and a second moment of the target frame image based on the trigger identifier. The time difference processing unit 203 is used to process data if the time difference between the first moment and the second moment is less than or equal to a preset deviation. If the threshold is reached, the target frame image is subjected to first inference processing based on the first inference path; the inference processing unit 204 is used to terminate the first inference processing if the estimated total time of the first inference processing is greater than the preset inference time limit, and perform second inference processing on the target frame image based on the second inference path to obtain the inference result; otherwise, the first inference processing is continued to obtain the inference result, wherein the computational resource consumption of the second inference processing is lower than that of the first inference processing; the data output unit 205 is used to output the inference data packet, which may include the trigger identifier, the first time, the second time and the inference result.

[0122] In some embodiments, the signal acquisition unit 201 is further configured to acquire the system timer count value when the trigger event occurs through a hardware interrupt service routine, determine the system timer count value as the first moment, and generate a globally unique sequence number as the trigger identifier based on the receiving order of the external trigger signal and according to a preset incrementing rule.

[0123] In some embodiments, the image acquisition unit 202 is further configured to retrieve a target frame image from the frame buffer queue of the image acquisition device, wherein the time difference between the frame capture timestamp and the first moment is the smallest; and determine the frame capture timestamp of the target frame image as the second moment.

[0124] In some embodiments, the time difference processing unit 203 is further configured to: if the time difference between the first time and the second time is greater than a preset deviation threshold, acquire the historical frame image corresponding to the previous trigger identifier adjacent to the trigger identifier; perform inter-frame difference processing on the target frame image and the historical frame image to obtain a difference image; determine whether the target frame image is a repeated trigger acquisition frame based on the distribution characteristics of non-zero pixels in the difference image; if it is determined to be a repeated trigger acquisition frame, use the historical inference result corresponding to the historical frame image as the inference result of the target frame image, and generate an inference data packet containing a reuse identifier; if it is determined to be a non-repeated trigger acquisition frame, generate a detection result containing an invalid state flag and a trigger identifier.

[0125] In some embodiments, the inference processing unit 204 is further configured to obtain the number of inference layers that have been completed by the preset artificial intelligence inference model and the total number of inference layers of the preset artificial intelligence inference model; determine the estimated remaining time to complete the remaining inference layers based on a first ratio of the number of completed inference layers to the total number of inference layers and the inference time consumed from the start of the first inference processing to the current time; obtain the current operating temperature of the edge computing terminal and the current load rate of the neural network processor; correct the estimated remaining time based on a first deviation between the current operating temperature and a preset reference temperature and a second deviation between the current load rate and a preset reference load rate to obtain a corrected estimated remaining time; and if the sum of the consumed inference time and the corrected estimated remaining time is greater than the preset inference time upper limit, send an inference termination instruction to the preset artificial intelligence inference model.

[0126] In some embodiments, the second inference path includes inference processing based on a preset lightweight detection algorithm, which is deployed in an independent hardware acceleration unit in the edge computing terminal. The independent hardware acceleration unit completes resource locking during the system initialization phase of the edge computing terminal so as not to be preempted by other tasks during operation.

[0127] In some embodiments, the edge computing inference device 20 further includes a data analysis unit, configured to obtain the time difference between a first moment and a second moment in multiple inference data packets within a preset statistical period; determine the trigger-frame synchronization jitter distribution within the preset statistical period based on the multiple time differences; and adjust the preset deviation threshold and / or the preset inference duration upper limit based on the trigger-frame synchronization jitter distribution.

[0128] In some embodiments, the data analysis unit is further configured to acquire product yield data of the production line corresponding to the edge computing terminal; construct a jitter-yield correlation model based on the trigger-frame synchronization jitter distribution and product yield data; predict the expected yield under different inference duration configurations in the next statistical period according to the jitter-yield correlation model; adjust the preset inference duration upper limit to the target inference duration, the target inference duration making the expected yield meet the preset yield constraint conditions and maximize the inference completion rate.

[0129] In some embodiments, the data output unit 205 is further configured to acquire the model version identifier, confidence information, degradation indication information, and image acquisition device identifier corresponding to the target frame image output by the first inference process or the second inference process; generate a data packet digest signature based on the unique device identifier and current timestamp of the edge computing terminal; and combine the model version identifier, confidence information, degradation indication information, image acquisition device identifier, data packet digest signature and trigger identifier, first time moment, second time moment, and inference result into an inference data packet.

[0130] This application also provides a computer-readable storage medium storing computer-executable instructions or a computer program, which, when executed by a processor, will cause the processor to perform any step of the edge computing inference method provided in this application.

[0131] In some embodiments, the computer-readable storage medium may be a random access memory (RAM), a read-only memory (ROM), flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); or it may be a variety of devices that include one or any combination of the above-mentioned memories.

[0132] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0133] In some embodiments, computer-executable instructions may, but do not necessarily, correspond to files in a file system, and may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0134] In some embodiments, computer-executable instructions may be deployed to execute on an electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0135] like Figure 3 As shown, this application also provides an electronic device 30, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements any step of the above-described edge computing inference method.

[0136] This application also provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium and executes the computer program or computer-executable instructions, causing the electronic device to perform any step of the edge computing inference method described above.

[0137] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An edge computing inference method, characterized in that, The edge computing inference method, applied to edge computing terminals, includes: Obtain the first moment and trigger identifier corresponding to the external trigger signal, wherein the external trigger signal is independent of the internal task scheduling clock of the edge computing terminal; Based on the trigger identifier, obtain the corresponding target frame image and the second moment of the target frame image; If the time difference between the first time point and the second time point is less than or equal to a preset deviation threshold, then the target frame image is subjected to first inference processing based on the first inference path; If the estimated total time of the first inference process is greater than the preset inference time limit during the first inference process, the first inference process is terminated, and the target frame image is subjected to second inference process based on the second inference path to obtain the inference result; otherwise, the first inference process is continued to obtain the inference result, wherein the computational resource consumption of the second inference process is lower than that of the first inference process. Output an inference data packet, the inference data packet including the trigger identifier, the first time point, the second time point, and the inference result.

2. The edge computing inference method according to claim 1, characterized in that, The acquisition of the first moment and trigger identifier corresponding to the external trigger signal includes: The system timer count value at the time of the triggering event is obtained through a hardware interrupt service routine, and the system timer count value is determined as the first moment. Based on the receiving order of the external trigger signals, a globally unique sequence number is generated according to a preset incrementing rule as the trigger identifier.

3. The edge computing inference method according to claim 1, characterized in that, The step of obtaining the corresponding target frame image and the second moment of the target frame image based on the trigger identifier includes: Retrieve the target frame image from the frame buffer queue of the image acquisition device, where the time difference between the frame capture timestamp and the first moment is the smallest. The frame capture timestamp of the target frame image is determined as the second moment.

4. The edge computing inference method according to claim 1, characterized in that, The edge computing inference method also includes: If the time difference between the first moment and the second moment is greater than the preset deviation threshold, then the historical frame image corresponding to the previous trigger identifier adjacent to the trigger identifier is obtained; Perform inter-frame difference processing on the target frame image and the historical frame image to obtain a difference image; Based on the distribution characteristics of non-zero pixels in the differential image, it is determined whether the target frame image is a repeatedly triggered acquisition frame; If it is determined that the acquisition frame is repeatedly triggered, the historical inference result corresponding to the historical frame image is used as the inference result of the target frame image, and an inference data packet containing a reuse identifier is generated. If the frame is determined to be a non-repeating trigger acquisition frame, a detection result containing an invalid status flag and the trigger identifier is generated.

5. The edge computing inference method according to claim 1, characterized in that, The step of terminating the first inference process if the estimated total time of the first inference process exceeds the preset inference time limit includes: Obtain the number of inference layers that have been completed by the preset artificial intelligence inference model and the total number of inference layers of the preset artificial intelligence inference model; Based on the first ratio of the number of inference layers that have been completed to the total number of inference layers, and the inference time consumed from the start of the first inference process to the current moment, the estimated remaining time to complete the remaining inference layers is determined. Obtain the current operating temperature of the edge computing terminal and the current load rate of the neural network processor; The estimated remaining time is corrected based on the first deviation between the current operating temperature and the preset reference temperature, and the second deviation between the current load rate and the preset reference load rate, to obtain the corrected estimated remaining time. If the sum of the consumed inference time and the revised estimated remaining time is greater than the preset inference time limit, then an inference termination instruction is sent to the preset artificial intelligence inference model.

6. The edge computing inference method according to claim 1, characterized in that, The second inference path includes inference processing based on a preset lightweight detection algorithm. The preset lightweight detection algorithm is deployed in an independent hardware acceleration unit in the edge computing terminal. The independent hardware acceleration unit completes resource locking during the system initialization phase of the edge computing terminal so as not to be preempted by other tasks during operation.

7. The edge computing inference method according to claim 1, characterized in that, Following the output inference data packet, the edge computing inference method further includes: Obtain the time difference between the first time point and the second time point in multiple inference data packets within a preset statistical period; Based on the multiple time differences, determine the trigger-frame synchronization jitter distribution within the preset statistical period; Based on the trigger-frame synchronization jitter distribution, the preset deviation threshold and / or the preset inference duration upper limit are adjusted.

8. The edge computing inference method according to claim 7, characterized in that, The dynamic adjustment of the preset deviation threshold and / or the preset inference duration upper limit based on the trigger-frame synchronization jitter distribution includes: Obtain the product yield data of the production line corresponding to the edge computing terminal; Based on the trigger-frame synchronization jitter distribution and the product yield data, a jitter-yield correlation model is constructed; Based on the jitter-yield correlation model, predict the expected yield under different inference duration configurations in the next statistical period; The preset inference time limit is adjusted to the target inference time, which makes the expected yield meet the preset yield constraint and maximize the inference completion rate.

9. The edge computing inference method according to any one of claims 1 to 8, characterized in that, The output inference data packet includes: Obtain the model version identifier, confidence information, downgrade indication information, and image acquisition device identifier corresponding to the target frame image output by the first inference process or the second inference process; Generate a data packet digest signature based on the unique device identifier and current timestamp of the edge computing terminal; The model version identifier, the confidence information, the downgrade indication information, the image acquisition device identifier, the data packet digest signature, the trigger identifier, the first time point, the second time point, and the inference result are combined into the inference data packet.

10. An edge computing inference device, characterized in that, The edge computing inference device, applied to an edge computing terminal, includes: The signal acquisition unit is used to acquire the first moment and trigger identifier corresponding to the external trigger signal, wherein the external trigger signal is independent of the internal task scheduling clock of the edge computing terminal; The image acquisition unit is used to acquire the corresponding target frame image and the second moment of the target frame image based on the trigger identifier; A time difference processing unit is used to perform a first inference processing on the target frame image based on a first inference path if the time difference between the first time and the second time is less than or equal to a preset deviation threshold. The inference processing unit is configured to terminate the first inference processing if the estimated total time of the first inference processing is greater than the preset inference time limit during the first inference processing, and perform a second inference processing on the target frame image based on the second inference path to obtain the inference result; otherwise, continue to execute the first inference processing to obtain the inference result, wherein the computational resource consumption of the second inference processing is lower than that of the first inference processing. The data output unit is used to output an inference data packet, which includes the trigger identifier, the first time point, the second time point, and the inference result.

Images