Food detection state visual output system and method based on machine vision

Abstract

The invention discloses a food detection state visual output system and method based on machine vision, and relates to the technical field of machine vision, and the system comprises a multi-source data synchronization module, a semantic analysis module, a visual rendering module, an interaction instruction processing module and an abnormity reinspection module. The multi-source data synchronization module generates a synchronization signal; the semantic analysis module extracts a multi-modal feature vector based on the synchronizing signal, and generates a feature vector which is matched with a user role and contains a semantic tag in combination with a pre-constructed lightweight knowledge graph; the visual rendering module dynamically generates an interactive interface according to semantic tags in the feature vectors; the interaction instruction processing module receives a positioning operation signal triggered by a user on an interaction interface to generate a traceability request instruction; and the exception rechecking module performs rechecking according to the traceability request instruction and the rendering instruction. According to the invention, the problems of insufficient surface defect detection precision, difficulty in quality tracing and lack of process closed-loop control in industrial production can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a food detection status visualization output system and method based on machine vision. Background Art

[0002] Traditional machine vision technology has been widely used in food inspection, for example, analyzing the sugar content of fruit through hyperspectral imaging, detecting foreign objects in packaged food using X-rays, and identifying surface defects in agricultural products through deep learning. Existing systems typically employ a step-by-step process: image acquisition undergoes preprocessing to extract features, followed by classification algorithms to output inspection results, ultimately presenting them as static reports or simply annotated images. However, with the expansion of inspection dimensions (such as the fusion of multi-source data such as ingredients, weight, and environmental parameters) and the diversification of user roles (quality inspectors, production managers, and supply chain regulators), traditional approaches have exposed significant shortcomings.

[0003] First, multimodal data has inconsistent temporal and spatial benchmarks due to the heterogeneity of acquisition equipment, making it difficult to accurately superimpose visible light images and near-infrared spectra and other data, affecting the accuracy of comprehensive analysis. Second, the test results are presented in a single way, with professionals relying on professional charts such as thermal maps and spectrum diagrams, which are difficult for non-technical personnel to understand intuitively, and static reports cannot support dynamic data tracing. In addition, abnormality tracing relies on manual retrieval of historical data for comparison, which is inefficient. Especially when tiny surface defects are detected, the operator cannot quickly trigger a local high-precision re-inspection to verify the results.

[0004] Although there are current studies attempting to introduce augmented reality technology, dynamic rendering is often delayed and stuck due to limitations in the computing power of embedded devices, and no adaptive association mechanism has been established between multi-source data and user behavior. For example, existing visualization systems are unable to automatically switch display dimensions based on user identity (e.g., production line operators need to locate defects in real time, while quality managers focus on batch compliance trends), resulting in information overload or omission of key data. At the same time, the interactive functions of most systems are limited to basic operations, and deep data analysis cannot be triggered through natural interactions (such as gestures and voice). The abnormality tracing process still requires manual operations across platforms, which reduces detection efficiency. In response to the above pain points, the present invention realizes full-link visualization and intelligent decision support for the detection process through spatiotemporal alignment of multi-source data, adaptive semantic mapping, and a closed-loop interactive traceability mechanism. Summary of the Invention

[0005] In view of the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a food inspection status visualization output system and method based on machine vision, which is used to solve the problems of difficulty in integrating multi-source data, poor visualization interactivity and low efficiency of abnormal traceability in food inspection. The present invention integrates visible light images, component sensor data and user identity information through a multi-source data synchronization module, and generates synchronization signals by using timestamp matching and spatial coordinate mapping; the semantic analysis module extracts multimodal features based on a lightweight knowledge graph and generates semantic tags adapted to user roles; the visualization rendering module dynamically generates an interactive interface based on semantic tags, and the defective area is enhanced by rendering triggered by compliance thresholds; the interactive instruction processing module parses user operations to generate traceability requests, the abnormal re-inspection module activates high-precision equipment for local re-inspection, and transmits feedback signals of the superimposed re-inspection results back to update the interface, forming a complete link of "data fusion-dynamic presentation-interactive traceability-closed-loop verification", realizing full-dimensional visualization of the inspection status and intelligent abnormal tracing.

[0006] The present invention provides a food detection status visualization output system based on machine vision, comprising: Multi-source data synchronization module, which receives visible light images, component sensor data, and user identity information, and generates synchronization signals through timestamp matching and spatial coordinate mapping; Semantic analysis module: The semantic analysis module extracts multimodal feature vectors based on synchronization signals and combines them with pre-built lightweight knowledge graphs to generate feature vectors containing semantic labels adapted to user roles. Visualization rendering module, which dynamically generates an interactive interface based on the semantic labels in the feature vector. The visualization level of the defect area is automatically triggered by the preset compliance threshold to enhance the rendering and generate rendering instructions; The interactive instruction processing module receives the positioning operation signal triggered by the user on the interactive interface, extracts the spatial coordinates of the target area and historical detection data to generate a traceability request instruction; The abnormal re-inspection module performs re-inspection according to the traceability request instruction and rendering instruction, and superimposes the re-inspection result with the original detection data to generate a traceability feedback signal, and then transmits the traceability feedback signal back to the visualization rendering module for dynamic update.

[0007] In one embodiment of the present invention, the multi-source data synchronization module establishes a unified spatial coordinate system through a laser calibration device, and establishes coordinate mapping rules with sub-millimeter accuracy between the visible light image and the component sensor data. When there is surface occlusion of the detection target, the spatial compensation parameters are generated based on the curvature characteristics of the edge contour of the occluded area, and the compensation parameters are written into the synchronization signal, so that the semantic analysis module can achieve precise alignment of the multimodal feature vectors according to the spatial coordinates after occlusion compensation.

[0008] In one embodiment of the present invention, the pre-built lightweight knowledge graph in the semantic analysis module includes a food quality standard library and a user role behavior pattern library. When receiving user identity information, the association weights of the semantic tags in the knowledge graph are dynamically adjusted by analyzing the interface residence time and function triggering frequency in the user's historical operation records, and an interpretable semantic description is generated based on the confidence distribution of the multimodal feature vector. The semantic tags include a hierarchical expression of the defect cause reasoning chain and the compliance judgment basis.

[0009] In one embodiment of the present invention, the visualization rendering module includes a dynamic color mapping strategy and a hierarchical focusing mechanism, which generates a progressive warning color scale based on the severity of the defect type in the semantic label. When it is detected that the user's gaze stays in a specific area of ​​the interactive interface for more than a threshold, the module automatically triggers the interpolation rendering of the local microscopic imaging data, while reducing the resolution of the non-focused area to optimize the allocation of computing resources. The rendering process is enhanced to maintain visual coherence through the optical flow estimation algorithm.

[0010] In one embodiment of the present invention, the interactive instruction processing module deploys a multimodal input parsing unit to support the coordinated operation of gesture trajectory recognition and voice commands. When the user defines the target area through the touch interface, the effective operation and the false touch signal are distinguished based on the acceleration characteristics of the fingertip motion trajectory, and the keyword extraction results in the voice command are integrated to correct the positioning range of the spatial coordinates. The positioning operation signal contains the abnormal confidence level marked by the user.

[0011] In one embodiment of the present invention, the abnormal re-inspection module includes a multi-stage verification strategy. After receiving the traceability request instruction, it first activates the wide-angle camera to quickly locate the target area, then switches to the microscope optical lens to collect the surface microstructure image, and finally controls the robotic arm to adjust the detection angle to obtain multi-view three-dimensional point cloud data. The re-inspection results are marked with morphological differences from the original data through a temporal difference algorithm, and a data credibility score is generated and embedded in the traceability feedback signal.

[0012] In one embodiment of the present invention, the system also includes a distributed rendering coordination unit. When the visualization rendering module detects the display performance parameters of the terminal device, it automatically splits the geometric modeling data and texture mapping data in the rendering instructions, and assigns the computationally intensive three-dimensional physical simulation tasks to the edge computing nodes. At the same time, it retains the real-time rendering capability of basic interface elements on the local device, and maintains the rendering status synchronization between multiple nodes through a bidirectional heartbeat signal.

[0013] In one embodiment of the present invention, a feature fusion channel is provided between the semantic analysis module and the visualization rendering module. When there is a sensor data conflict in the multimodal feature vector, a feature weighting coefficient is generated based on the industry standard priority rules in the knowledge graph, and the contribution of different data sources in the semantic label is dynamically adjusted through the attention mechanism. The correction records generated by the conflict resolution process will be fed back to the multi-source data synchronization module for calibrating subsequent acquisition parameters.

[0014] In one embodiment of the present invention, the interactive instruction processing module is connected to a historical data warehouse. When generating a traceability request instruction, the historical inspection records of the same batch of products are extracted based on the spatial coordinates of the target area, and a probability prediction model is generated by comparing the evolution trend of the defect morphology. The prediction result is injected into the traceability feedback signal in the form of a virtual overlay layer, so that the dynamically updated visual interface can simultaneously display the current inspection status and potential risk warning information.

[0015] The present invention also provides a method for visually outputting food detection status based on machine vision, comprising: S1: Receives visible light images, component sensor data, and user identity information, and generates synchronization signals through timestamp matching and spatial coordinate mapping; S2: Extract multimodal feature vectors based on synchronization signals and combine them with pre-built lightweight knowledge graphs to generate user-role-adapted feature vectors containing semantic labels. S3: Dynamically generates an interactive interface based on the semantic labels in the feature vector, where the visualization level of the defect area is automatically triggered by the preset compliance threshold to enhance the rendering and generate rendering instructions; S4: Receive the positioning operation signal triggered by the user on the interactive interface, extract the spatial coordinates of the target area and historical detection data to generate a traceability request instruction; S5: Re-inspect according to the traceability request instruction and rendering instruction, and superimpose the re-inspection result with the original detection data to generate a traceability feedback signal, and then transmit the traceability feedback signal back to the visualization rendering module for dynamic update.

[0016] The machine vision-based food inspection status visualization output system and method provided by the present invention integrate visible light images, component sensor data and user identity information through a multi-source data synchronization module, and use timestamp matching and spatial coordinate mapping to generate synchronization signals; the semantic analysis module extracts multimodal features based on a lightweight knowledge graph and generates semantic labels adapted to user roles; the visualization rendering module dynamically generates an interactive interface based on semantic labels, and enhanced rendering of defective areas is triggered by compliance thresholds; the interactive instruction processing module parses user operations to generate traceability requests, and the abnormal re-inspection module activates high-precision equipment for local re-inspection, and transmits feedback signals with superimposed re-inspection results back to update the interface, forming a complete link of "data fusion-dynamic presentation-interactive traceability-closed-loop verification", thereby realizing full-dimensional visualization of the inspection status and intelligent abnormal tracing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 This is the system architecture diagram of the food inspection status visualization output system based on machine vision; Figure 2 The flowchart of the method for visual output of food inspection status based on machine vision. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0020] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0021] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0022] See Figure 1-2 , shown is a food inspection status visualization output system and method based on machine vision of the present invention. The food inspection status visualization output system based on machine vision of the present invention includes a multi-source data synchronization module, a semantic analysis module, a visualization rendering module, an interactive instruction processing module and an abnormal re-inspection module. The multi-source data synchronization module receives visible light images, component sensor data and user identity information, and generates a synchronization signal through timestamp matching and spatial coordinate mapping; the semantic analysis module extracts multimodal feature vectors based on the synchronization signal, and generates a feature vector containing semantic labels adapted to the user role in combination with a pre-built lightweight knowledge graph; the visualization rendering module dynamically generates an interactive interface based on the semantic labels in the feature vector, wherein the visualization level of the defective area is automatically triggered by the preset compliance threshold to enhance rendering, and generates rendering instructions; the interactive instruction processing module receives the positioning operation signal triggered by the user on the interactive interface, extracts the spatial coordinates of the target area and the historical detection data to generate a traceability request instruction; the abnormal re-inspection module performs re-inspection according to the traceability request instruction and the rendering instruction, and superimposes the re-inspection result with the original detection data to generate a traceability feedback signal, and transmits the traceability feedback signal back to the visualization rendering module for dynamic update.

[0023] like Figure 1As shown, the present invention relates to a food inspection status visualization output system based on machine vision. The multi-source data synchronization module is the underlying data fusion core for constructing the visualization system. Its technical implementation includes three key levels: first, a laser calibration device is deployed at the hardware level to perform three-dimensional spatial modeling of the inspection area by emitting a laser grid of a specific wavelength. The device is physically calibrated with devices such as visible light cameras and near-infrared component sensors to establish a unified spatial coordinate system with sub-millimeter accuracy, so that the data collected by different sensors can be superimposed in the spatial dimension. When the detection target is obscured by the surface (for example, the wrinkles of a food packaging bag cover part of the area), the module generates spatial compensation parameters by analyzing the curvature characteristics of the edge contour of the obscured area and combining it with the complete surface morphology library in the historical collected data. Specifically, the Bezier surface fitting algorithm is used to reconstruct the missing area in three dimensions, and the compensation parameters are written into the metadata field of the synchronization signal to ensure that subsequent modules can perform accurate analysis based on the corrected coordinates. In terms of time synchronization, the module features a built-in high-precision clock chip that timestamps the data streams of all connected devices with microsecond-level timestamps. A sliding window algorithm eliminates inter-device acquisition delays. The resulting synchronization signal not only includes spatial coordinate mappings but also embeds sensor calibration coefficients and data confidence assessments, providing a reliable data baseline for subsequent semantic analysis. This module's innovation lies in its breakthrough in the traditional, simple, parallel processing of multi-source data. It achieves deep cross-modal data fusion through spatiotemporal coupling technology. For example, it can precisely overlay surface cracks from visible light images with water loss data from near-infrared spectroscopy on a three-dimensional model, revealing the correlation patterns underlying defect generation.

[0024] Furthermore, the semantic analysis module undertakes the core conversion function from raw data to knowledge expression, and its technical architecture includes two major subsystems: feature extraction engine and dynamic knowledge graph. The feature extraction engine adopts a multi-branch neural network structure to process the texture features of visible light images, the spectral features of component sensors, and the behavioral features of user identities. The image branch adopts an improved residual attention network, which adds a channel attention mechanism on the basis of the traditional convolutional layer, focusing on capturing the local features of small defects; the spectral branch extracts frequency domain features through wavelet transform to eliminate sensor noise interference. The dynamic knowledge graph is built based on industry standards and historical cases, and includes three knowledge domains: food quality compliance rule base, user role permission tree, and equipment operation parameter set. When receiving user identity information, the module dynamically adjusts the connection weights between knowledge nodes by analyzing the interface interaction heat map and function call records in the user's historical operations. For example, it strengthens the semantics related to real-time defect location for production line inspectors and highlights the statistical dimension of batch compliance rate for quality managers. When generating semantic labels, the module uses a probabilistic graphical model to fuse and calculate the confidence of multimodal feature vectors. When conflicting detection data exists (e.g., visible light images appear normal while near-infrared data is abnormal), an arbitration mechanism is initiated based on the priority rules in the knowledge graph. For example, in a moisture detection scenario, near-infrared sensors are given a higher weight. Interpretable descriptive text containing conflict markers is also generated. The output format of the semantic labels adopts a multi-layered nested structure. The first layer is the intuitive conclusion (e.g., "82% probability of surface mold"), the second layer expands the judgment basis (including the association of sensor data and knowledge item number), and the third layer provides actionable suggestions (e.g., "It is recommended to initiate the microscopic re-inspection process"). This hierarchical representation meets the need for rapid decision-making while maintaining a complete logical traceability chain. By establishing a mapping relationship between data features and semantic space, the module effectively lowers the understanding threshold for non-expert users while providing structured input for visual rendering.

[0025] Specifically, the visual rendering module is the core vehicle for intelligent human-computer interaction. Its technical solution encompasses two major innovations: dynamic rendering strategies and resource optimization mechanisms. At the dynamic rendering level, the module invokes a predefined color mapping rule library based on the defect type and severity level in the semantic label. For example, it uses a red gradient for microbial contamination and an orange pulse warning for physical damage. The intensity of the color gradient dynamically adjusts with the defect probability value. When the module detects that the user's gaze remains within a specific area of ​​the interactive interface for longer than a set threshold, it initiates a hierarchical focusing mechanism: first, it obtains the gaze coordinates using an eye-tracking device and maps them to the corresponding position on the 3D model. It then uses local microscopic imaging data (such as the 40x magnified image provided by the re-inspection module) for interpolated rendering. Super-resolution reconstruction technology is used to enhance detail clarity, while resolution downsampling and texture compression are implemented in non-focused areas, allowing computing resources to focus on critical areas. To ensure visual coherence, the module incorporates an optical flow estimation algorithm to predict the user's gaze trajectory, preloads texture data for adjacent areas, and employs dynamic blur technology to eliminate screen tearing caused by rendering delays. The resource optimization mechanism is reflected in the distributed rendering architecture: the module monitors the GPU performance and memory usage of the terminal device in real time. When processing complex three-dimensional models, the geometric modeling data is split into a basic mesh and a detailed displacement map. The basic mesh is rendered on the local device, and the detailed map is processed in parallel through the edge computing node, and finally seamless splicing is achieved through streaming technology. For different display devices such as mobile terminals and AR glasses, the module has a built-in adaptive layout engine that can dynamically adjust the scale and position of interface elements according to the screen size and viewing angle range. For example, in the AR field of view, key parameters are presented as floating cards in real-time detection data, while on the desktop terminal, panoramic three-dimensional models and trend analysis charts are displayed. Through the above-mentioned technological innovations, the module has achieved stable output of 4K-level visualization effects under limited hardware resources, while supporting multi-terminal collaborative operations and real-time data synchronization, significantly improving the efficiency of human-computer collaboration in the detection process.

[0026] In one embodiment of the present invention, the interactive command processing module is the core hub of human-computer interaction, and its technological innovation is reflected in the deep integration of multimodal input parsing and intelligent signal filtering. The parsing unit deployed by this module integrates a gesture trajectory recognition engine and a speech semantic analysis model, and realizes parallel signal processing through a multi-threaded architecture. Gesture recognition uses an improved spatiotemporal convolutional network to decompose the motion trajectory of the user's finger on the touch interface into a position sequence and an acceleration feature vector, where the acceleration feature is used to distinguish between valid operations and false touches: when a sudden change in the direction of fingertip movement is detected and the acceleration peak exceeds the natural operation range (such as a sudden jitter or sliding pause), the false touch filtering mechanism is automatically triggered, the abnormal trajectory segment is discarded, and a continuous and smooth operation signal is retained. Voice command processing adopts an end-to-end speech recognition framework, extracts Mel-frequency cepstral coefficients through an acoustic model, and combines the food inspection professional vocabulary in the language model for semantic decoding. The noise reduction algorithm is optimized specifically for the noisy environment of the production line, and beamforming technology is used to enhance the target voice signal. When the user defines the target area by touch, the module jointly analyzes the gesture coordinates and the directional descriptors in the voice command (such as "the moldy area in the upper left corner"), and uses the semantic similarity algorithm to correct the spatial positioning range: for example, when the user mentions "the third apple in the second row" by voice, the system automatically performs coordinate weighted fusion of the rough area defined by the gesture and the logical position indicated by the voice, and finally generates an accurate positioning signal. The abnormal confidence level embedded in the positioning operation signal is derived from the pressure sensor data and the dwell time analysis during the user operation. For example, in the AR glasses interaction scenario, if the user stares at an area for more than a set time threshold, the confidence level of the area will be automatically increased, triggering a high-priority re-inspection process. The innovative value of this module lies in breaking through the traditional single interaction mode and enhancing the accuracy of operation intention recognition through the complementarity of multimodal signals. For example, when the voice command is vague in a noisy environment, the acceleration characteristics of the gesture trajectory and the pressure data can still be used to comprehensively judge the user's actual operation target.

[0027] like Figure 1As shown, the abnormality re-inspection module has built a multi-level verification system, and its technical implementation includes two major subsystems: equipment collaborative control and data difference analysis. At the physical execution level, the module connects a wide-angle camera, a microscope optical lens, and a six-degree-of-freedom robotic arm through an industrial bus protocol to form an adaptive detection chain: upon receiving a traceability request instruction, it first controls the wide-angle camera to scan the target area at a rate of 30 frames per second, using a feature matching algorithm to quickly locate the abnormal coordinates; then the microscope head moves along the guide rail to directly above the target, using a piezoelectric ceramic micro-displacement platform to achieve submicron-level focusing, and using a multi-band LED light source (including UV excitation and polarized light modes) to capture surface microstructure images, such as identifying the degree of cell rupture on the fruit skin or the distribution of pores in baked goods; finally, the robotic arm carries a three-dimensional laser scanning head and moves around the detection target from multiple perspectives, acquiring millimeter-level precision point cloud data through structured light projection. During this process, a dynamic path planning algorithm is used to avoid collisions with production line equipment. At the data analysis level, the module uses a temporal difference algorithm to compare the re-inspection data with the original inspection results: it calculates the inter-frame differences of continuously collected microscopic images, extracts morphological change features (such as crack propagation speed and moldy area growth rate), and simultaneously performs ICP (iterative closest point) registration on the three-dimensional point cloud data and the original model to calculate curvature changes and volume differences. The traceability feedback signal finally generated not only contains the comparison conclusion between the re-inspection results and the original data, but also embeds a data credibility score, which is calculated based on the sensor calibration records, lighting condition stability and algorithm confidence. For example, when the signal-to-noise ratio of the microscopic image decreases in a low-light environment, the system automatically reduces the credibility weight of the re-inspection result. This module ensures the objectivity and reliability of the abnormality tracing conclusion through the dual protection of hardware collaboration and algorithm verification.

[0028] Furthermore, the distributed rendering coordination unit addresses the technical challenges of cross-platform visualization. Its core innovation lies in the dynamic partitioning of rendering tasks and multi-node state synchronization. The unit incorporates a built-in device performance probe that collects real-time information about the device's GPU model, video memory capacity, and network bandwidth. Upon detecting a mobile device, it automatically activates the texture compression pipeline: high-resolution textures are converted to BC7 compression format and LOD (level of detail) simplification is applied to the geometry, preserving critical vertex data while removing minor triangles. For complex scenes requiring physical simulation (such as the flow of liquid food within a container), the unit splits the computational tasks into basic dynamics calculations and high-order particle effects rendering. The former is solved using finite element methods at the edge computing node, while the latter performs real-time fluid simulation on a cloud-based GPU cluster. The rendered results are ultimately streamed to the device via the WebGL 2.0 standard. Regarding synchronization, the unit utilizes a bidirectional heartbeat signal to maintain state consistency across nodes. The master node sends a timestamp-containing synchronization packet to its child nodes every 200 milliseconds, and the child nodes respond with the current rendering progress and resource utilization. If a failure to respond for more than three heartbeat cycles is detected, the unit automatically initiates a fault-tolerant rendering process, reassigning the task to a backup node. This unit specifically optimizes transmission protocols for weak network environments, employs adaptive bitrate technology to dynamically adjust data stream resolution, and automatically switches to key feature wireframe mode when network bandwidth falls below 10Mbps, ensuring the visual interface remains operational. Through these technologies, the system achieves seamless cross-device collaboration while preserving visual quality. For example, while a quality inspector views a simplified 2D heat map on a tablet, an engineer's workstation can simultaneously display a 3D model including physical simulation.

[0029] like Figure 2 The figure shows a method for visualizing the food inspection status based on machine vision according to the present invention. S1: Receive visible light images, component sensor data, and user identity information, and generate a synchronization signal through timestamp matching and spatial coordinate mapping; S2: Extract multimodal feature vectors based on the synchronization signal, and generate a feature vector containing semantic tags adapted to the user role in combination with a pre-built lightweight knowledge graph; S3: Dynamically generate an interactive interface based on the semantic tags in the feature vector, wherein the visualization level of the defective area is automatically triggered by a preset compliance threshold for enhanced rendering, and a rendering instruction is generated; S4: Receive a positioning operation signal triggered by the user on the interactive interface, extract the spatial coordinates of the target area and historical detection data to generate a traceability request instruction; S5: Re-inspect according to the traceability request instruction and the rendering instruction, and superimpose the re-inspection result with the original detection data to generate a traceability feedback signal, and transmit the traceability feedback signal back to the visualization rendering module for dynamic update.

[0030] like Figure 2As shown, the feature fusion channel builds a closed-loop system for data conflict resolution and knowledge iteration. Its technical implementation includes three stages: conflict detection, weight allocation, and feedback calibration. When the semantic analysis module receives the multimodal feature vector, the channel starts the consistency verification process: the spatial overlap analysis is performed on the defect area identified by the visible light image and the component abnormality area detected by the near-infrared spectrum. If the coordinate deviation between the two exceeds the preset threshold, it is marked as a data conflict. At this time, based on the predefined industry standard priority rules in the knowledge graph (such as microbial detection takes precedence over appearance detection in food safety standards), a feature weighting coefficient matrix is ​​generated. For example, in the meat detection scenario, a higher weight is given to X-ray foreign body detection data. At the same time, the attention mechanism is introduced to dynamically adjust the feature contribution: a spatial attention map is constructed through a deformable convolutional network, focusing on areas with high consistency in multi-sensor data, and reducing the feature fusion intensity for conflicting areas. The correction records generated during the conflict resolution process contain the sensor ID, conflict type, and the arbitration rules used. These records are transmitted to the multi-source data synchronization module via a feedback interface, triggering adaptive calibration of acquisition parameters. For example, when frequent data conflicts occur between the visible light camera and the near-infrared sensor under specific lighting conditions, the camera's white balance parameters or the near-infrared sensor's integration time are automatically adjusted, reducing data inconsistencies at the source. The innovation of this channel lies in establishing a positive feedback loop from data fusion to acquisition optimization. For example, if systematic deviations are discovered between a certain type of weighing sensor and visual volume estimation data during long-term operation, calibration coefficients are automatically generated and firmware is updated to continuously improve the system's overall detection accuracy.

[0031] The machine vision-based food inspection status visualization output system and method of the present invention integrates visible light images, component sensor data and user identity information through a multi-source data synchronization module, and uses timestamp matching and spatial coordinate mapping to generate synchronization signals; the semantic analysis module extracts multimodal features based on a lightweight knowledge graph and generates semantic labels adapted to user roles; the visualization rendering module dynamically generates an interactive interface based on semantic labels, and enhanced rendering of defective areas is triggered by compliance thresholds; the interactive instruction processing module parses user operations to generate traceability requests, and the abnormal re-inspection module activates high-precision equipment for local re-inspection, and transmits feedback signals with superimposed re-inspection results back to update the interface, forming a complete link of "data fusion-dynamic presentation-interactive traceability-closed-loop verification", realizing full-dimensional visualization of the inspection status and intelligent abnormal tracing.

[0032] Therefore, the food inspection status visualization output system and method based on machine vision of the present invention solves the problems of difficulty in integrating multi-source data, poor visualization interactivity and low efficiency of abnormal traceability in food inspection.

[0033] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A food inspection status visualization output system based on machine vision, characterized in that: include: A multi-source data synchronization module receives visible light images, component sensor data, and user identity information, and generates a synchronization signal through timestamp matching and spatial coordinate mapping; A semantic analysis module, which extracts a multimodal feature vector based on the synchronization signal and generates a feature vector containing semantic tags adapted to the user role in combination with a pre-built lightweight knowledge graph; A visualization rendering module, which dynamically generates an interactive interface based on the semantic tags in the feature vector, wherein the visualization level of the defect area is automatically triggered by a preset compliance threshold to enhance rendering and generates rendering instructions; An interactive instruction processing module, which receives a positioning operation signal triggered by a user on the interactive interface, extracts the spatial coordinates of the target area and historical detection data, and generates a traceability request instruction; An abnormality recheck module, which performs a recheck based on the traceability request instruction and the rendering instruction, and superimposes the recheck result with the original detection data to generate a traceability feedback signal, and transmits the traceability feedback signal back to the visualization rendering module for dynamic update.

2. The food detection status visualization output system based on machine vision according to claim 1 is characterized in that: The multi-source data synchronization module establishes a unified spatial coordinate system through a laser calibration device, and establishes coordinate mapping rules with sub-millimeter accuracy between the visible light image and the component sensor data. When the detection target is obscured by the surface, the spatial compensation parameters are generated based on the curvature characteristics of the edge contour of the obscured area, and the compensation parameters are written into the synchronization signal, so that the semantic analysis module can achieve precise alignment of the multimodal feature vectors according to the spatial coordinates after occlusion compensation.

3. The food detection status visualization output system based on machine vision according to claim 1, characterized in that: The pre-built lightweight knowledge graph in the semantic analysis module includes a food quality standard library and a user role behavior pattern library. When receiving user identity information, the association weights of the semantic tags in the knowledge graph are dynamically adjusted by analyzing the interface residence time and function triggering frequency in the user's historical operation records, and an interpretable semantic description is generated based on the confidence distribution of the multimodal feature vector. The semantic tags include a hierarchical expression of the defect cause reasoning chain and the compliance judgment basis.

4. The food detection status visualization output system based on machine vision according to claim 1, characterized in that: The visualization rendering module includes a dynamic color mapping strategy and a hierarchical focusing mechanism, generating a progressive warning color scale based on the severity of the defect type in the semantic label. When it is detected that the user's gaze stays in a specific area of ​​the interactive interface for longer than a threshold, it automatically triggers interpolation rendering of local microscopic imaging data, while reducing the resolution of non-focused areas to optimize computing resource allocation. The enhanced rendering process maintains visual coherence through an optical flow estimation algorithm.

5. The food detection status visualization output system based on machine vision according to claim 1 is characterized in that: The interactive command processing module is deployed with a multimodal input parsing unit, which supports the coordinated operation of gesture trajectory recognition and voice commands. When the user defines the target area through the touch interface, the module distinguishes between valid operations and false touch signals based on the acceleration characteristics of the fingertip motion trajectory, and integrates the keyword extraction results in the voice command to correct the positioning range of the spatial coordinates. The positioning operation signal contains the abnormality confidence level marked by the user.

6. The food detection status visualization output system based on machine vision according to claim 1, characterized in that: The abnormal re-inspection module includes a multi-stage verification strategy. After receiving the traceability request instruction, it first activates the wide-angle camera to quickly locate the target area, then switches to the microscope optical lens to collect surface microstructure images, and finally controls the robotic arm to adjust the detection angle to obtain multi-view three-dimensional point cloud data. The re-inspection results are marked with morphological differences from the original data through a temporal difference algorithm, and a data credibility score is generated and embedded in the traceability feedback signal.

7. The food detection status visualization output system based on machine vision according to claim 1 is characterized in that: The system also includes a distributed rendering coordination unit. When the visualization rendering module detects the display performance parameters of the terminal device, it automatically splits the geometric modeling data and texture mapping data in the rendering instructions, and allocates computationally intensive three-dimensional physical simulation tasks to edge computing nodes. At the same time, it retains the real-time rendering capability of basic interface elements on the local device and maintains the rendering status synchronization between multiple nodes through a bidirectional heartbeat signal.

8. The food detection status visualization output system based on machine vision according to claim 1, characterized in that: A feature fusion channel is provided between the semantic analysis module and the visualization rendering module. When there is a sensor data conflict in the multimodal feature vector, a feature weighting coefficient is generated based on the industry standard priority rules in the knowledge graph, and the contribution of different data sources in the semantic label is dynamically adjusted through the attention mechanism. The correction records generated by the conflict resolution process will be fed back to the multi-source data synchronization module for calibrating subsequent acquisition parameters.

9. The food detection status visualization output system based on machine vision according to claim 1, characterized in that: The interactive instruction processing module is connected to a historical data warehouse. When generating a traceability request instruction, it extracts historical inspection records of products from the same batch based on the spatial coordinates of the target area, generates a probability prediction model by comparing the evolution trend of defect morphology, and injects the prediction results into the traceability feedback signal in the form of a virtual overlay, so that the dynamically updated visual interface simultaneously displays the current inspection status and potential risk warning information.

10. A method for visually outputting food inspection status based on machine vision according to any one of claims 1 to 9, characterized in that: include: S1: Receives visible light images, component sensor data, and user identity information, and generates synchronization signals through timestamp matching and spatial coordinate mapping; S2: extracting a multimodal feature vector based on the synchronization signal, and generating a feature vector containing semantic labels adapted to the user role by combining it with a pre-built lightweight knowledge graph; S3: Dynamically generate an interactive interface based on the semantic tags in the feature vector, wherein the visualization level of the defect area is automatically triggered by a preset compliance threshold to enhance rendering, and generate rendering instructions; S4: receiving a positioning operation signal triggered by the user on the interactive interface, extracting the spatial coordinates of the target area and historical detection data to generate a traceability request instruction; S5: re-checking according to the traceability request instruction and the rendering instruction, and superimposing the re-checking result with the original detection data to generate a traceability feedback signal, and transmitting the traceability feedback signal back to the visualization rendering module for dynamic update.

Images