An online detection method based on machine vision and near-infrared fusion

Abstract

The application discloses an online detection method based on machine vision and near-infrared fusion, and relates to the technical field of optical intelligent sensing, which comprises the following steps: inputting a multi-band feature fusion image into a defect evolution prediction model, dynamically modeling color change, texture change and light transmittance gradient change of a local area, identifying potential defect development direction, rate and evolution range, forming a defect evolution time sequence characteristic vector, and giving a local area a priority processing index according to the defect expansion rate; and performing spatial integration and coordinate normalization to generate a complete defect distribution map. Through the defect evolution prediction model, the color change, the texture change and the light transmittance gradient change are continuously and dynamically modeled, and the local area priority processing index is formed according to the defect expansion rate, so that the appearance abnormalities and the internal light transmittance response of the traditional Chinese medicine decoction pieces are associated and expressed in the same time sequence chain, and the identification stability of the potential defect development direction, the change rate and the evolution range is improved.

Description

Technical Field

[0001] This invention relates to the field of optical intelligent sensing technology, and in particular to an online detection method based on the fusion of machine vision and near-infrared spectroscopy. Background Technology

[0002] The processing of Chinese medicinal herbs is gradually moving towards continuous, digital, and online quality control. Machine vision can acquire information on the color, shape, and texture of the surface of the herbs, while near-infrared detection can reflect the internal light transmission characteristics, changes in tissue structure, and component-related responses of the material. These detection methods have high application value in monitoring the quality of Chinese medicinal herb processing.

[0003] Existing online detection methods focus on single-frame image recognition and single spectral feature determination. There is a lack of continuous correlation between surface property detection results and near-infrared internal response data, making it difficult to form a stable temporal judgment under changes in transport, turning, and processing conditions. As a result, the development process of appearance abnormalities, internal changes, and defects cannot be expressed synchronously. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an online detection method based on the fusion of machine vision and near-infrared spectroscopy to solve the problem of the difficulty in synchronous, real-time and non-destructive monitoring of appearance and internal components during the processing of traditional Chinese medicine.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides an online detection method based on machine vision and near-infrared fusion, comprising: continuously acquiring visible light and near-infrared images of each piece of traditional Chinese medicine decoction along a transport path, and dynamically adjusting the illumination angle of the light source according to the offset and blur caused by the transport motion to form a vibration-compensated image; performing pixel-level fusion on the vibration-compensated image, and analyzing the color distribution, texture features, and light transmittance features of each local region to identify local microstructural differences and potential abnormal regions, generating a multi-band feature fusion image; inputting the multi-band feature fusion image into a defect evolution prediction model to dynamically model the color changes, texture changes, and light transmittance gradient changes of local regions, and identify potential defect development paths. The defect evolution time-series feature vector is formed by considering the direction, rate, and range of the defect expansion. Local regions are assigned priority processing indices based on the defect expansion rate. Based on the defect evolution time-series feature vector and local region priority processing indices, color anomalies, texture anomalies, and light transmission anomalies in local regions of vibration-compensated images and multi-band feature fusion images are prioritized and multi-dimensionally fused to generate aggregated defect scoring indices for each local region. These aggregated defect scoring indices are mapped to the spatial coordinates of each Chinese herbal medicine slice. Connectivity analysis and boundary fusion are performed on adjacent and overlapping local regions to obtain composite defect regions. All composite defect regions are then spatially integrated and their coordinates normalized to generate a complete defect distribution map.

[0007] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the specific steps for forming the vibration-compensated image are as follows: The surface color and internal structure information of each slice of medicine will be recorded by acquiring continuous visible light images and near-infrared images to obtain multimodal image data; Based on the image offset and blur caused by the conveying motion, the multimodal image data is dynamically aligned and reconstructed to extract the position information and local texture features of each pixel and obtain the vibration correction image after displacement compensation. By utilizing local feature changes in vibration-corrected images and dynamically adjusting the illumination angle of the light source, a vibration-compensated image is formed through illumination compensation and continuous frame fusion.

[0008] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the specific steps for generating the multi-band feature fusion image are as follows: The color, texture and light transmission information of each pixel are obtained from the vibration-compensated image, and normalization is performed to eliminate the acquisition bias of the image to obtain local feature data. Pixel-level fusion of local feature data is performed. By analyzing the feature differences of each local region, color, texture and light transmission information are recombined in multiple bands to obtain a fused multi-band local feature map. Anomaly identification and microstructure difference assessment are performed on the fused multi-band local feature maps to generate a multi-band feature fusion image.

[0009] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the defect evolution prediction model is specifically constructed as follows: A defect evolution prediction model is constructed based on a multimodal local feature encoding layer, a defect evolution time series analysis layer, and a defect evolution feature generation layer. The multimodal local feature coding layer extracts, classifies, and normalizes the color, texture, and light transmission information of local regions in the multi-band feature fusion image to generate a set of local feature data. The defect evolution time-series analysis layer, based on multi-band feature fusion images, performs continuous frame sorting and time series indexing on local feature data sets, and generates local regional time-series evolution information through difference analysis, trend statistics and dynamic cumulative calculation. The defect evolution feature generation layer calculates a local region priority processing index based on the local region temporal evolution information, uses the local region priority processing index as an intermediate input, and performs multi-dimensional fusion and deduction with the local region temporal evolution information to generate a defect evolution temporal feature vector.

[0010] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the specific steps for forming the defect evolution time-series feature vector are as follows: Based on multi-band feature fusion images, the color changes, texture features and light transmission features of each local region are extracted and classified to obtain a local feature data set, and a time series index is established by sorting consecutive frames; Based on the time series index, continuous frame comparison and trend analysis are performed on the local feature data set to generate local area color change trend, texture change trend and transmittance gradient change results, identify potential abnormal areas, and calculate the development direction and change rate of potential defects through cumulative analysis to form the temporal evolution information of the local area. By utilizing temporal evolution information, the rate of defect expansion in each local region is evaluated, a local region priority processing index is generated, and the priority processing index is combined with the color, texture and transmittance change trends to form a defect evolution temporal feature vector.

[0011] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the step of performing continuous frame comparison and trend analysis on the local feature data set according to the time series index to generate local region color change trend, texture change trend and transmittance gradient change results, identifying potential abnormal regions, and calculating the development direction and change rate of potential defects through cumulative analysis to form the temporal evolution information of the local region includes performing difference analysis on the local feature data set of continuous frames, and identifying candidate regions with obvious abnormal change amplitudes by comparing the color, texture and transmittance changes of each local region; The candidate regions are evaluated for cumulative trends on the time series index, and the parts of the local region that show continuous abnormal changes in consecutive frames are identified as potential abnormal regions. Extract the dynamic evolution of potential abnormal regions, calculate the possible development direction and rate of change of defects, and form the results into time-series evolution information of local regions.

[0012] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the specific steps for generating the aggregated defect scoring index for each local region are as follows: Based on the time-series feature vector of defect evolution and the local region priority processing index, color anomalies, texture anomalies and light transmission anomalies of each local region in the vibration compensation image and the multi-band feature fusion image are extracted, and each anomaly feature is normalized to obtain a local feature dataset. Based on the local feature dataset, the abnormal features of each local region are sorted according to the priority processing index, and the sorting results are fused and recombined with the temporal evolution information to generate multi-dimensional abnormal feature combination data for each local region. Based on multi-dimensional abnormal feature combination data, combined with local color abnormality, texture abnormality and light transmission abnormality information and defect evolution time sequence feature vector, it is determined whether there are defects in each local area, and an aggregated defect scoring index for each local area is generated.

[0013] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the step of determining whether there are defects in each local area based on multi-dimensional abnormal feature combination data, combined with local color abnormality, texture abnormality and light transmission abnormality information and defect evolution time sequence feature vector, and generating aggregated defect scoring index for each local area includes analyzing the multi-dimensional abnormal feature combination data of each local area and identifying potential abnormal areas by comparing the change range of color abnormality, texture abnormality and light transmission abnormality. By combining the identified potential abnormal regions with the defect evolution time-series feature vector, the probability of defects occurring and their evolution trend in local areas are assessed, and an aggregate defect score index for each local area is generated based on the assessment results.

[0014] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the specific steps of mapping the aggregated defect scoring index to the spatial coordinates of each piece of traditional Chinese medicine decoction piece, and performing connectivity analysis and boundary fusion on adjacent and overlapping local regions to obtain the composite defect region are as follows: The spatial coordinates of each Chinese herbal medicine slice are mapped using the aggregation defect scoring index, and adjacent and overlapping local areas are extracted. Through boundary analysis and merging, integrated candidate defect areas are obtained. Based on the regional structure of the integrated candidate defect regions, all integrated candidate defect regions are spatially integrated and their coordinates are normalized, and then uniformly mapped according to their spatial location to obtain the composite defect region.

[0015] As a preferred embodiment of the online detection method based on machine vision and near-infrared fusion described in this invention, the step of spatially integrating and coordinate normalizing all composite defect regions to generate a complete defect distribution map is as follows: Based on the spatial coordinates of each local area mapped to the aggregated defect scoring index, the positions and boundaries of each composite defect area are summarized and aligned. By smoothing the boundaries of adjacent areas and adjusting their spatial positions, a set of integrated composite defect area data is obtained. Based on the integrated composite defect region dataset, the location and boundary data of each composite defect region are normalized, the normalized composite defect regions are mapped to a spatial coordinate system, and the regional data are reorganized to generate a complete defect distribution map.

[0016] The beneficial effects of this invention are as follows: by using a defect evolution prediction model, continuous dynamic modeling is performed on color changes, texture changes and transmittance gradient changes, and a local area priority processing index is formed based on the defect expansion rate, so that the appearance abnormalities of Chinese herbal medicine slices and internal light transmittance response are associated and expressed in the same time-series chain, thereby improving the identification stability of the development direction, change rate and evolution range of potential defects. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an online detection method based on the fusion of machine vision and near-infrared spectroscopy.

[0019] Figure 2 This is a flowchart for image acquisition compensation and multi-band fusion.

[0020] Figure 3 This is a flowchart for the defect evolution prediction model analysis.

[0021] Figure 4 A flowchart is generated to link defect scoring and defect distribution. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides an online detection method based on machine vision and near-infrared fusion, comprising the following steps: S1: Continuous visible light and near-infrared images are acquired for each piece of Chinese herbal medicine along the conveying path. The illumination angle of the light source is dynamically adjusted according to the offset and blur caused by the conveying motion to form a vibration-compensated image.

[0026] It should be noted that the local region is an image processing unit divided based on the outline range and pixel spatial coordinates of each piece of Chinese herbal medicine. The imaging area of ​​each piece of Chinese herbal medicine is determined according to the outline of the piece after alignment with the visible light image and near-infrared image, and the imaging area is used as the local region division range. Within the imaging area, the region is divided according to a sliding window of a fixed size. The window size is determined according to the image acquisition resolution and the actual size of the piece of Chinese herbal medicine, and the value is from 16×16 pixels to 64×64 pixels, preferably 32×32 pixels. The window step size is from half the window side length to one window side length, so that there is partial overlap between adjacent local regions, ensuring that boundary defects and small defects are not missed due to region segmentation.

[0027] For windows located at the edge of the herbal slice outline, valid pixels falling within the herbal slice outline are retained, and edge windows with less than half the number of valid pixels are merged into adjacent local regions. Each local region is configured with a region identifier, frame number, region center coordinates, region boundary coordinates, and internal pixel set, and a mapping relationship is established with the spatial coordinates of the corresponding herbal slice. Adjacent local regions refer to local regions whose region boundaries are connected, whose center coordinate spacing does not exceed a preset step size, or whose internal pixel sets have a continuous adjacency relationship. Overlapping local regions refer to local regions whose outer boundaries overlap or whose internal pixel sets have a common coverage area.

[0028] S1.1: Collect surface color and internal structure information of each slice of medicine by acquiring continuous visible light images and near-infrared images to obtain multimodal image data.

[0029] Furthermore, the Chinese herbal medicine slices to be tested are sent one by one to the image acquisition position along the transport path. The acquisition sequence number of the slices is established according to the order in which they enter the acquisition position. Continuous visible light images and near-infrared images are acquired simultaneously when the same slice passes through the acquisition position.

[0030] The surface color, outer edge morphology, and surface texture information of each slice of medicinal food are extracted from continuous visible light images. The light transmittance response and internal structure information of each slice of medicinal food are extracted from near-infrared images. According to the acquisition sequence number and acquisition time, the continuous visible light images, near-infrared images, surface color information, and internal structure information corresponding to the same slice of medicinal food are organized to form a set of visible light and near-infrared data for each slice of medicinal food. By arranging and integrating all the data sets of medicinal food according to the transport path sequence, multimodal image data is obtained.

[0031] Specifically, the light transmittance response information and internal structure information of each slice of Chinese herbal medicine are extracted from near-infrared images. Near-infrared images are used to reflect the changes in absorption, scattering and light transmittance response of Chinese herbal medicine slices in the near-infrared band. The light transmittance response information is not only used to characterize the differences in the internal structure of the slices, but also to characterize the spectral response changes related to moisture content, extract content and active ingredients online. When the water content, internal tissue looseness or component-related absorption response of the slices changes during the processing, the corresponding near-infrared light transmittance intensity, light transmittance distribution and light transmittance boundary will change, and together with the surface color, texture and morphology information in the visible light image, they form multimodal image data.

[0032] S1.2: Based on the image offset and blur caused by the conveying motion, the multimodal image data is dynamically aligned and reconstructed to extract the position information and local texture features of each pixel and obtain the vibration correction image after displacement compensation.

[0033] Furthermore, based on the image offset and blurring caused by the transport motion, when dynamically aligning and reconstructing multimodal image data, the visible light image at the same acquisition time is used as the spatial alignment reference. The near-infrared image is grayscaled, edge enhanced, and contour extracted to obtain the contour boundary of each Chinese herbal medicine slice in the near-infrared image. Based on the contour boundary of each Chinese herbal medicine slice in the visible light image and the contour boundary of the corresponding slice in the near-infrared image, the edge points, center points, and local texture similar regions of the slices are matched to establish the pixel correspondence between the visible light image and the near-infrared image.

[0034] Based on pixel correspondence, the pixel position changes of the same local area in consecutive frames are tracked to determine the offset direction and range caused by the transport motion. By comparing the changes in local texture clarity in consecutive frames, the blurred areas caused by the transport motion are identified. According to the offset direction, offset range, and blurred areas, the visible light pixel position, near-infrared pixel position, and local texture position in the multimodal image data are synchronously corrected so that the surface color information, near-infrared light transmission information, and local texture features of the same Chinese herbal medicine slice correspond to the same spatial position. For local pixels that are missing or misaligned after correction, interpolation is performed to complete them based on the color information, light transmission information, and texture continuity of adjacent pixels. The completed visible light color information, near-infrared light transmission information, and local texture features are then rearranged according to the corrected spatial position to obtain a vibration-corrected image after displacement compensation.

[0035] S1.3: By utilizing the local feature changes of the vibration-corrected image, the illumination angle of the light source is dynamically adjusted, and a vibration-compensated image is formed through illumination compensation and continuous frame fusion.

[0036] Furthermore, the vibration-corrected image after displacement compensation is divided into multiple local regions according to the outline of the medicinal slices and the pixel position. The brightness center, texture boundary position and light transmission intensity distribution of each local region in consecutive frames are extracted. The brightness center offset, texture boundary offset and light transmission intensity fluctuation of the same local region in adjacent frames are aligned and compared to obtain the direction and magnitude of local feature changes.

[0037] The compensation direction of the light source illumination angle is determined based on the direction of local feature changes, and the adjustment amount of the light source illumination angle is determined based on the magnitude of local feature changes. The light source illumination direction is adjusted in the opposite direction of the image offset, and illumination compensation is performed on areas with uneven brightness and uneven transmittance. The consecutive frames after illumination compensation are then aligned a second time according to the pixel position information. For pixels with clear boundaries, color information, texture information, and transmittance information are preserved. For pixels with blurred boundaries, pixels with consistent positions and continuous textures in adjacent frames are fused and completed. The fused and completed image is then subjected to local brightness equalization and boundary continuity adjustment to form a vibration-compensated image.

[0038] It should be noted that the dynamic adjustment of the light source illumination angle is not based solely on changes in the overall brightness of the image, but is determined by the combined shifts in brightness center, texture boundary, and light transmission intensity distribution center of the same local area in the vibration-corrected image across consecutive frames.

[0039] By comparing the brightness center position, texture boundary position, and light transmission intensity distribution center position of the same local area in adjacent frames, the offset direction and offset distance of the brightness center, texture boundary, and light transmission intensity distribution center are obtained. When the offset direction is consistent or has a corresponding relationship in consecutive frames, it is used as the image offset direction and the direction of local feature change. The amplitude of local feature change is determined based on the offset distance. The compensation direction of the light source illumination angle is determined by the direction of local feature change, and the compensation amount of the light source illumination angle is determined by the amplitude of local feature change. The light source illumination direction is adjusted in the opposite direction of the image offset direction, and the compensated illumination area is consistent with the actual imaging area of ​​the slice.

[0040] The direction of local feature change is jointly determined by the offset direction of the brightness center, the offset direction of the texture boundary, and the offset direction of the light transmission intensity distribution in the same local area in consecutive frames. When the brightness center, texture boundary, and light transmission intensity distribution are offset in the same direction in consecutive frames, the direction is determined as the image offset direction, and the light source illumination direction is adjusted to the opposite direction to the image offset direction. The amplitude of local feature change is obtained by combining the offset distance of the brightness center, the offset distance of the texture boundary, and the fluctuation range of the light transmission intensity. The larger the amplitude of local feature change, the greater the degree of adjustment of the corresponding light source illumination angle, so that the compensated illumination area is consistent with the actual imaging area of ​​the slice.

[0041] The formula for compensating for the illumination angle of the light source is: ; in, This indicates the current record of the same local region of the same Chinese herbal medicine slice in a continuous frame sequence. This indicates the previous frame of the current record. and These represent the brightness center positions of local areas in the current record and the previous frame record, respectively. and These represent the texture boundary positions of local regions in the current record and the previous frame record, respectively. and These represent the center positions of the light transmittance distribution in local areas of the current and previous frames, respectively. This indicates the calibrated distance from the reference position illuminated by the light source to the imaging plane of the Chinese herbal medicine slices. This represents the arctangent mapping function, used to convert the ratio of local feature offset to calibration distance into an angle. This indicates the amount of compensation for the illumination angle of the light source corresponding to the current local area. The negative sign indicates that the illumination direction of the light source is adjusted in the opposite direction to the image offset direction.

[0042] S2: Perform pixel-level fusion on the vibration compensation image, analyze the color distribution, texture features and light transmission features of each local region, identify local microstructure differences and potential abnormal regions, and generate a multi-band feature fusion image.

[0043] S2.1: Obtain the color information, texture information and light transmission information of each pixel based on the vibration compensation image, and normalize it to eliminate the acquisition deviation of the image and obtain local feature data.

[0044] Furthermore, a spatial coordinate index for each pixel is established based on the vibration-compensated image. The color data of the visible light channel and the grayscale data of the near-infrared channel in the vibration-compensated image are read according to the spatial coordinate index. The color data of the visible light channel is converted to color space to extract the color information corresponding to each pixel. At the same time, a neighborhood window is selected with each pixel as the center, and the grayscale changes, edge changes and grayscale distribution uniformity within the neighborhood window are statistically analyzed to obtain the texture information corresponding to each pixel.

[0045] Transmittance intensity is extracted from the grayscale data of the near-infrared channel to obtain the transmittance information corresponding to each pixel. Acquisition deviation correction is performed on the color information, texture information, and transmittance information respectively. Abnormal extreme values ​​caused by exposure, brightness fluctuations, and local reflections in the same type of information are smoothed and linearly normalized according to the numerical distribution range of the same type of information to convert the color information, texture information, and transmittance information into data of a uniform scale. According to the spatial coordinate index of each pixel, the normalized color information, texture information, and transmittance information are correlated to form local feature data containing pixel position, color information, texture information, and transmittance information.

[0046] Specifically, the acquisition bias includes local brightness fluctuations in visible light images, differences in surface reflection of medicinal slices, and fluctuations in light transmittance in near-infrared images. When normalizing color, texture, and light transmittance information, the color distribution range, texture distribution range, and light transmittance distribution range of each pixel within the same medicinal slice are sorted out separately. Abnormal extreme values ​​that exceed the continuous change trend of adjacent areas are smoothed out, and the processed color, texture, and light transmittance information are converted to a unified data scale so that multiple features at the same pixel location can be fused accordingly.

[0047] Transmittance information includes near-infrared transmittance intensity, transmittance gradient, local absorption response, and transmittance distribution uniformity. Transmittance information is used to characterize the internal moisture distribution, extract content changes, and near-infrared absorption characteristics changes of effective components in the medicinal slices. When the same local area shows continuous changes in transmittance intensity, abrupt changes in transmittance gradient, or deviations in local absorption response in consecutive frames, it indicates that there may be changes in the internal moisture, extract, or effective components in the local area. These changes are then used as near-infrared internal response characteristics to participate in multi-band feature fusion.

[0048] For example, in the online detection process of Chinese herbal medicine slices after drying, steaming, stir-frying or baking, near-infrared images can be used to monitor the residual moisture state, changes in the distribution of extracts, and changes in the absorption response of effective components in different local areas of the slices. When a local area experiences a continuous decrease in near-infrared light transmittance, an increase in local absorption response, a sudden change in transmittance gradient, or uneven light transmittance distribution due to insufficient drying, uneven processing, or changes in internal structure, the area is recorded as an area with abnormal internal quality response. This area is then analyzed in correspondence with color changes, texture changes, and morphological changes in visible light images to achieve synchronous online monitoring of the appearance and internal quality of Chinese herbal medicine slices.

[0049] S2.2: Perform pixel-level fusion of local feature data. By analyzing the feature differences of each local region, color, texture and light transmission information are recombined in multiple bands to obtain a fused multi-band local feature map.

[0050] Furthermore, the local feature data are arranged according to the pixel coordinates in the vibration-compensated image, and the color information, texture information, and light transmission information corresponding to each pixel are extracted. The color information, texture information, and light transmission information under the same pixel coordinates are matched to obtain the pixel-level correspondence. Based on the pixel-level correspondence, the color depth changes, texture continuity changes, and light transmission intensity changes of adjacent pixels and adjacent local regions in each local region are compared to obtain the feature differences of each local region.

[0051] Based on the feature differences of each local region, color, texture and light transmission information are recombined in multiple bands according to the same pixel coordinates. Color information is used to characterize surface color differences, texture information is used to characterize surface structure differences, and light transmission information is used to characterize near-infrared internal response differences. The recombined color, texture and light transmission information are synchronously written into the corresponding local region to obtain the fused multi-band local feature map.

[0052] S2.3: Perform anomaly identification and microstructure difference assessment on the fused multi-band local feature map to generate a multi-band feature fusion image.

[0053] Furthermore, based on pixel location, color, texture, and light transmission information are read from the fused multi-band local feature map. The color changes, texture continuity changes, and light transmission intensity changes of adjacent pixels in the same local area are summarized to obtain the feature difference results of each local area. The feature difference results of each local area are compared with the feature difference results of adjacent local areas. Local areas with abrupt changes in color distribution, interruption of texture continuity, or abrupt changes in light transmission characteristics are identified as potential anomalous areas. Adjacent pixels in potential anomalous areas are connected and merged to form continuous anomalous areas.

[0054] The positions of color boundaries, texture boundaries, and light transmission boundaries in continuous abnormal regions are verified. The parts of color boundaries, texture boundaries, and light transmission boundaries that fall into the same local region are identified as local microstructure difference positions. These local microstructure difference positions are then backfilled into the corresponding pixel positions of the fused multi-band local feature map, so that color information, texture information, and light transmission information are marked with anomalies and microstructure differences in the same spatial location, generating a multi-band feature fusion image.

[0055] It should be noted that anomaly identification and microstructure difference assessment refer to the process of extracting the spatial positions of color boundaries, texture boundaries, and light transmission boundaries within potential anomaly regions when assessing microstructure differences in potential anomaly regions. The degree of overlap and proximity of the three types of boundaries within the same local region is compared. When color boundaries, texture boundaries, and light transmission boundaries overlap, are close to, or change continuously within the same local region, the corresponding pixel position is determined as the local microstructure difference position. When only a single boundary changes and does not form a corresponding relationship with the boundary, the pixel position is retained as a single anomaly marker and is not directly determined as the local microstructure difference position.

[0056] S3: Input the multi-band feature fusion image into the defect evolution prediction model, dynamically model the color change, texture change and transmittance gradient change of the local area, identify the development direction, rate and evolution range of potential defects, form a defect evolution time-series feature vector, and assign priority processing index to the local area according to the rate of defect expansion; the defect evolution prediction model is constructed based on the multimodal local feature encoding layer, the defect evolution time-series analysis layer and the defect evolution feature generation layer.

[0057] S3.1: Multimodal local feature coding layer, which extracts, classifies and normalizes the color, texture and light transmission information of local areas of multi-band feature fusion image to generate local feature data set.

[0058] S3.2: Defect Evolution Time Series Analysis Layer. Based on multi-band feature fusion images, it performs continuous frame sorting and time series indexing on local feature data sets. Through difference analysis, trend statistics and dynamic cumulative calculation, it generates local regional time series evolution information.

[0059] S3.3: Defect Evolution Feature Generation Layer. Based on the temporal evolution information of the local region, it calculates the local region priority processing index, uses the local region priority processing index as an intermediate input, and performs multi-dimensional fusion and deduction with the local region temporal evolution information to generate the defect evolution temporal feature vector.

[0060] First, continuous visible light and near-infrared images of Chinese herbal medicine slices under different processing stages, different appearance conditions, and different internal light transmittance response conditions are collected. Then, vibration-compensated images and multi-band feature fusion images are generated according to the aforementioned processing procedure. Color changes, texture changes, light transmittance gradient changes, regional center positions, and regional boundary positions of each local region are extracted from the multi-band feature fusion images and organized into a local feature data set and a time series index according to the continuous frame order.

[0061] The local feature data set is input into the multimodal local feature encoding layer. The color change, texture change, and transmittance gradient change of the same local region in consecutive frames are input into the defect evolution temporal analysis layer. The local region temporal evolution information and the local region priority processing index calculated from the temporal evolution information are input into the defect evolution feature generation layer. The defect evolution feature generation layer outputs the corresponding defect evolution temporal feature vector. The output result is compared with the training label, and the defect region position deviation, defect development direction deviation, change rate deviation, evolution range deviation, and aggregate defect score deviation are calculated respectively. The deviation is used as the training error evaluation content of the defect evolution prediction model. Based on the error evaluation results, the parameters of the multimodal local feature encoding layer, defect evolution temporal analysis layer, and defect evolution feature generation layer are corrected round by round to reduce the deviation between the next round output result and the training label.

[0062] The number of training samples is determined based on the type of medicinal slices to be detected, the processing stage, and the coverage of defect types. In one embodiment, the training samples are no less than 500 sets of continuous frame samples, and the verification samples are no less than 100 sets of continuous frame samples. The training stopping condition of the defect evolution prediction model is that, in multiple consecutive rounds of training, the defect area identification results, defect development direction judgment results, change rate judgment results, evolution range judgment results, and aggregated defect score results of the verification samples all meet the preset verification requirements, and the error decrease is less than the change amplitude in multiple consecutive rounds of training. Stability means that the defect evolution prediction model can continuously output defect areas, defect development directions, change rates, and evolution ranges consistent with the results of manual review or processing quality inspection in the verification samples, so that the output defect evolution time sequence feature vector prioritizes the generation of indexes and aggregated defect score indicators in local areas.

[0063] S3.4: Based on multi-band feature fusion images, the color changes, texture features and light transmission features of each local region are extracted and classified to obtain a set of local feature data, and a time series index is established by sorting consecutive frames.

[0064] Furthermore, the multi-band feature fusion image is divided into local regions according to the spatial coordinates of each piece of Chinese herbal medicine, and a region identifier and frame number are configured for each local region. Within the same local region, the color information, texture information and near-infrared light transmission information of the corresponding positions of the current frame and the previous frame are read. After color space conversion of the color information, the color channel differences, brightness differences and hue shift of the corresponding local regions are compared to obtain the quantification results of color changes in the local regions.

[0065] The texture information is analyzed for edge density, gray-level distribution, local gray-level variance, and texture continuity. Changes in edge continuity, texture coarseness, and texture distribution uniformity are compared within corresponding local regions to obtain quantified texture features. Similarly, near-infrared transmittance information is analyzed for transmittance intensity, transmittance gradient, and transmittance boundary position. Differences in near-infrared transmittance intensity, transmittance distribution, and transmittance boundary offset are compared within corresponding local regions to obtain quantified transmittance features. Using the value ranges of the quantification results of color change, texture feature, and light transmission feature of all local areas within the same piece of Chinese herbal medicine as normalization benchmarks, the color change, texture feature, and light transmission feature of each local area are converted to a unified data scale for continuous frame comparison, trend analysis, and judgment of abnormal change amplitude.

[0066] The extraction results are categorized and organized according to region identifier, frame number, color change, texture features, and light transmission features. Feature data belonging to the same local region and from consecutive frames are arranged in the same data group to obtain a local feature data set. The local feature data set is then sorted into consecutive frames according to the image acquisition time and frame number. The color change, texture features, and light transmission features of each local region are recorded in chronological order to create a time series index.

[0067] S3.5: Perform differential analysis on the local feature data set of consecutive frames, and identify candidate regions with significant abnormal changes by comparing the changes in color, texture and transmittance of each local region.

[0068] Furthermore, based on the time series index, the same local area of ​​the same piece of Chinese herbal medicine in adjacent consecutive frames is matched in position, and the color information, texture information and transmittance information of the corresponding local area are extracted, and color continuous frame data, texture continuous frame data and transmittance continuous frame data are formed respectively. Taking the color change amplitude, texture change amplitude and transmittance change amplitude of each local area in the current time series index as the comparison object, the change amplitude of each local area between adjacent frames is calculated and accumulated frame by frame to obtain local area color difference data, local area texture difference data and local area transmittance difference data.

[0069] Color difference data, texture difference data, and transmittance difference data formed in the same local area between adjacent time frames are organized into three categories of difference data. These are then sorted and compared with the corresponding difference data of all local areas in the current time series index. When a local area is continuously in the abnormal sorting range for at least one of the difference data in color difference, texture difference, or transmittance difference, and the abnormal position of the local area maintains coordinate correspondence or boundary overlap in multiple adjacent time frames, it is determined that the local area has a continuous deviation from the overall change level of the current medicinal slices. The continuity of the continuously deviating local area is further confirmed. Local areas that simultaneously meet the criteria of continuous frame abnormality, abnormal position correspondence, and consistent feature change direction are identified as candidate areas with obvious abnormal change amplitude.

[0070] Specifically, the abnormal sorting interval is determined by sorting the color difference data, texture difference data, and transmittance difference data of all local regions within the current time series index. Local regions with high change amplitudes and repeated occurrences in consecutive frames are included in the abnormal sorting interval. If the same local region enters the abnormal sorting interval only in a single time frame and the time frame does not maintain abnormal changes, the local region is excluded as an instantaneous fluctuation region. If the same local region is continuously in the abnormal sorting interval in multiple adjacent time frames and the abnormal position remains corresponding, it is determined as a candidate region with obvious abnormal change amplitude.

[0071] Candidate regions with significant abnormal changes are determined by comparing the change amplitudes of all local regions within the current time series index. For the same piece of Chinese herbal medicine, the color change amplitude, texture change amplitude, and transmittance change amplitude are calculated for all local regions within the current time series index, and color difference data sequences, texture difference data sequences, and transmittance difference data sequences are formed respectively. The difference data are sorted from high to low according to the change amplitude.

[0072] When a local region ranks in the top N% of all local regions in terms of change magnitude in at least one of the sorting results of color difference data, texture difference data, or transmittance difference data, or when the difference data corresponding to the local region is greater than M times the average magnitude of the same type of difference data in the current time series index, and the local region maintains coordinate correspondence, boundary overlap, or consistent feature change direction in adjacent consecutive frames, the local region is identified as a candidate region with obvious abnormal change magnitude.

[0073] The value of N is determined based on the sorted distribution of all local area difference data in the current batch of tests, ranging from 5% to 20%. M is determined based on the average variation of normal medicinal slices samples or non-abnormal local areas in the current batch, ranging from 1.5 to 3.0. When the surface texture difference of the tested object is large or the transport vibration is strong, N takes a smaller value or M takes a larger value to reduce the instantaneous fluctuations being misjudged as candidate areas. When the defect of the tested object is small or the light transmission is abnormally weak, N takes a larger value or M takes a smaller value.

[0074] S3.6: Perform cumulative trend evaluation on the time series index for candidate regions, and identify the parts of the local region that show continuous abnormal changes in consecutive frames as potential abnormal regions.

[0075] Furthermore, local feature data sets of the same local region in adjacent consecutive frames are extracted according to the time series index. Using the spatial coordinates of the local region as the correspondence, the color change, texture change, and transmittance gradient change of the candidate region in each frame are arranged in frame order to obtain the continuous change record of the candidate region. The inter-frame difference of the continuous change record of the candidate region is accumulated. The color change, texture change, and transmittance gradient change are judged frame by frame to determine whether the color change, texture change, and transmittance gradient change remain in a continuous change state. The number of consecutive frames, change direction, and change amplitude corresponding to the continuous change state are recorded to obtain the cumulative change result of the candidate region. Based on the cumulative change result of the candidate region, local regions that show abnormal changes only in a single frame or a few discontinuous frames are filtered out. Local regions that show continuous color change, texture change, and transmittance gradient change in consecutive frames or at least one change shows a continuous enhancement feature are retained. The retained local regions are matched with the time series index, and the parts of the local region that show continuous abnormal changes in consecutive frames are identified as potential abnormal regions.

[0076] S3.7: Extract the dynamic evolution of potential abnormal regions, calculate the possible development direction and rate of change of defects, and form the results into the temporal evolution information of local regions.

[0077] Furthermore, color change data, texture change data, transmittance gradient change data, region center position, region boundary position, and corresponding acquisition time of the same potential anomaly region in each consecutive frame are extracted from the local feature data set. The above data are arranged according to the time series index to obtain the continuous frame evolution data of the potential anomaly region. The time frame in which the potential anomaly region is first confirmed is taken as the starting frame. The region center position and region boundary position in the starting frame are used as the evolution benchmark. The region center position offset, boundary expansion position, boundary contraction position, and boundary overlap range in subsequent adjacent time frames are compared frame by frame to determine the positional change relationship of the potential anomaly region in the transport path direction, lateral edge direction, or local diffusion direction.

[0078] Using the center and boundary positions of the potential anomaly region in the initial frame as a benchmark, the center position offset direction, center position movement distance, boundary expansion direction, boundary expansion distance, and boundary expansion range are extracted in subsequent adjacent time frames. When the center position offset direction is consistent with or adjacent to the boundary expansion direction, they are jointly determined as the development direction of the potential defect. When the directions are inconsistent, the continuity of the center position movement and the persistence of the boundary expansion range are compared. If the center position maintains the same offset direction in multiple consecutive frames, the center position offset direction is taken as the main development direction. If the boundary expansion range continues to increase in multiple consecutive frames, the boundary expansion direction is taken as the main development direction.

[0079] The rate of change is determined based on the center position movement distance, boundary expansion distance, and corresponding acquisition time interval between adjacent time frames. When the actual size is not calibrated, the rate of change is expressed in pixels per frame or pixels per second. When the image pixel and the actual size calibration relationship of the slice has been established, the rate of change is converted to actual length per second. When processing the center position movement distance and boundary expansion distance, if the potential abnormal area mainly exhibits overall migration, the rate of change is determined mainly by the center position movement distance. If the potential abnormal area mainly exhibits boundary expansion, the rate of change is determined mainly by the boundary expansion distance. If both overall migration and boundary expansion exist, the center position movement distance and boundary expansion distance are fused and processed according to their stability and expansion contribution in consecutive frames to obtain the comprehensive rate of change of the potential abnormal area.

[0080] The development direction, rate of change, regional boundary change results, color change trend, texture change trend, and transmittance gradient change results of potential defects are grouped together according to the same local area to form the temporal evolution information of the local area.

[0081] S3.8: Utilize temporal evolution information to evaluate the rate of defect expansion in each local region, generate local region priority processing index, and combine the priority processing index with the color, texture, and transmittance change trends to form a defect evolution temporal feature vector.

[0082] Furthermore, by utilizing temporal evolution information, the location, color change trend, texture change trend, and transmittance change trend of potential abnormal regions in each local area in consecutive frames are read. The change order of each local area in adjacent frames is arranged according to the time series index. Based on the arranged consecutive frame data, the boundary displacement change, color change duration, texture change duration, and transmittance change duration of potential abnormal regions in each local area are statistically analyzed to obtain the continuous change data of defect expansion in each local area.

[0083] Based on the continuously changing data of defect expansion, the rate of defect expansion in each local region in consecutive frames is evaluated, and each local region is sorted from high to low according to the rate of defect expansion, generating a local region priority processing index. The local region priority processing index is then recombined with the color change trend, texture change trend, and transmittance change trend, so that each local region simultaneously contains the defect expansion speed, color change direction, texture change direction, and transmittance change direction, forming a defect evolution temporal feature vector.

[0084] The formula for the local area priority processing index is: ; in, This indicates the number of frames in which the current local region has continuously exhibited abnormal changes up to the current record, used to characterize the persistence of the abnormal changes. The rate of defect propagation in the current local area is derived from the changes in the center position and boundary expansion of the potential anomaly region. The intensity of abnormal changes in the current local area is represented by a normalized and processed result of color changes, texture changes, and transmittance gradient changes. The extent of defect evolution in the current local area is represented by the sum of the potential anomaly region boundary expansion range and the anomaly region coverage range in consecutive frames. This indicates the priority indicators for processing the current local area.

[0085] The local region priority processing index formula incorporates the number of frames of anomaly duration, defect development rate, anomaly change intensity, and defect evolution range into the priority processing judgment, so that local regions with long continuous anomaly duration, fast expansion speed, high anomaly intensity, and large evolution range can obtain higher priority processing indexes, which are used as inputs for anomaly feature ranking and aggregated defect scoring.

[0086] Before participating in the calculation of the local area priority processing index, the potential defect development rate and defect evolution range are normalized according to the value range of all local area corresponding data in the current time series index and converted into dimensionless values ​​in the range of 0 to 1. The number of abnormal duration frames and the intensity of abnormal change are also normalized according to the same scale to keep the various parameters in the local area priority processing index formula consistent in terms of dimensions.

[0087] Specifically, the local region priority processing index indicates the degree to which the same local region should be given priority in defect scoring in consecutive frames. The local region priority processing index is obtained by combining the defect expansion rate, the duration of abnormal changes, and the expansion range of abnormal regions. When the defect expansion rate of a local region is fast, the abnormal changes persist in consecutive frames, and the boundary of the abnormal region continues to expand, the corresponding local region receives a higher priority processing index. When a local region only shows a brief change or the abnormal region does not continue to expand, the corresponding local region receives a lower priority processing index.

[0088] S4: Based on the temporal feature vector of defect evolution and the local region priority processing index, the local region color anomalies, texture anomalies and light transmission anomalies of vibration compensation images and multi-band feature fusion images are prioritized and multi-dimensionally fused and deduced to generate aggregated defect scoring indexes for each local region.

[0089] S4.1: Based on the temporal feature vector of defect evolution and the local region priority processing index, extract the color anomaly, texture anomaly and light transmission anomaly of each local region in the vibration compensation image and the multi-band feature fusion image, and normalize each anomaly feature to obtain the local feature dataset.

[0090] Furthermore, based on the local region location, color change trend, texture change trend, and transmittance gradient change results recorded in the defect evolution time sequence feature vector, the corresponding local region in the vibration compensation image and the multi-band feature fusion image is located. The processing order of each local region is determined according to the local region priority processing index. Color distribution data and texture feature data of the corresponding local region are extracted from the vibration compensation image, and transmittance feature data of the corresponding local region are extracted from the multi-band feature fusion image. Color distribution data is analyzed for color deviation to obtain color anomalies, texture feature data is analyzed for texture continuity and edge change to obtain texture anomalies, and transmittance feature data is analyzed for transmittance gradient change to obtain transmittance anomalies. Then, the color anomalies, texture anomalies, and transmittance anomalies of each local region are organized according to the local region location, so that the three types of anomaly features of the same local region form a set of local anomaly feature data. Each local anomaly feature data is normalized to convert the color anomalies, texture anomalies, and transmittance anomalies into a unified data range. The normalized local anomaly feature data is sorted and summarized according to the local region priority processing index to obtain the local feature dataset.

[0091] Specifically, when normalizing color anomalies, texture anomalies, and light transmission anomalies, the minimum and maximum values ​​of the corresponding anomaly data for all local areas within the same piece of Chinese herbal medicine slice are used as the linear normalization benchmark. The color anomalies, texture anomalies, and light transmission anomalies of each local area are mapped to a unified data range of 0 to 1. The closer the normalized value is to 1, the higher the degree of anomaly in the corresponding anomaly type of the local area; the closer the normalized value is to 0, the lower the degree of anomaly in the corresponding anomaly type of the local area. When the maximum and minimum values ​​of the same type of anomaly data are the same, it indicates that the anomaly of this type has no distinguishing effect within the current slice. The normalization result of the anomaly of this type is uniformly set to 0 to avoid anomaly calculation results caused by a zero denominator, and to ensure that color anomalies, texture anomalies, and light transmission anomalies can participate in sorting, summarizing, and aggregating defect scoring index generation at the same scale.

[0092] S4.2: Based on the local feature dataset, sort the abnormal features of each local region according to the priority processing index, and merge and reorganize the sorting results with the time-series evolution information to generate multi-dimensional abnormal feature combination data for each local region.

[0093] Furthermore, color anomalies, texture anomalies, and light transmission anomalies of each local region are extracted from the local feature dataset. Priority processing indicators, color change trends, texture change trends, light transmission gradient change results, potential defect development direction, and change rate of the same local region are extracted from the defect evolution time sequence feature vector. Using the location number of the local region as the corresponding basis, the color anomalies, texture anomalies, and light transmission anomalies of each local region are associated and arranged with the corresponding priority processing indicators. The local regions are then sorted from high to low according to the priority processing indicators to obtain the sorted local anomaly feature sequence.

[0094] Then, based on the time series index, the sorted local anomaly feature sequence is matched with the temporal evolution information of the same local region in consecutive frames, so that the static anomaly features of each local region correspond to the dynamic evolution features. The priority processing indicators, color anomalies, texture anomalies, light transmission anomalies, color change trends, texture change trends, light transmission gradient change results, potential defect development direction and change rate of the same local region are organized in a unified data order to generate multi-dimensional anomaly feature combination data for each local region.

[0095] S4.3: Analyze the multi-dimensional abnormal feature combination data of each local area, and identify potential abnormal areas by comparing the changes in color abnormality, texture abnormality and light transmission abnormality.

[0096] Furthermore, the local color anomaly, texture anomaly, and light transmission anomaly information of the same local region in consecutive frames are read according to the time series index. The differences in color anomaly changes, texture anomaly changes, and light transmission anomaly changes corresponding to the current frame and the previous frame are extracted to obtain the amplitude of the three types of anomaly changes in each local region in consecutive frames. The amplitudes of the three types of anomaly changes in the same local region are accumulated and sorted in chronological order, retaining the continuously enhanced color anomaly changes, texture anomaly changes, and light transmission anomaly changes, and removing instantaneous fluctuations that only appear in a single frame and do not continue in subsequent frames. The amplitudes of the three types of anomaly changes that are continuously enhanced are compared in combination within the same region. When at least two types of anomalies (color anomaly, texture anomaly, and light transmission anomaly) or the amplitude of light transmission anomaly changes continuously enhances in the same local region in consecutive frames, and is accompanied by a synchronous enhancement of either color anomaly or texture anomaly, the corresponding local region is marked as a potential anomaly region, thus completing the identification of potential anomaly regions.

[0097] S4.4: Combine the identified potential abnormal regions with the defect evolution time sequence feature vector to assess the probability of defect occurrence and evolution trend in local regions, and generate an aggregated defect score index for each local region based on the assessment results.

[0098] Furthermore, based on multi-dimensional abnormal feature combination data, local color abnormality information, local texture abnormality information, and local light transmission abnormality information corresponding to the identified potential abnormal regions are extracted according to local region numbers. The potential abnormal regions are then aligned with the color change trend, texture change trend, light transmission gradient change result, potential defect development direction, and change rate of the same local regions in the defect evolution time-series feature vector to obtain abnormal feature alignment data of potential abnormal regions.

[0099] Based on the alignment data of abnormal features, the variation amplitudes of local color abnormality information, local texture abnormality information, and local light transmission abnormality information are sorted. Combined with the potential defect development direction and change rate in the defect evolution time sequence feature vector, the abnormality persistence, abnormal expansion, and abnormal change intensity of each local region are marked item by item to obtain local region defect assessment data. Based on the local region defect assessment data, the abnormality persistence, abnormal expansion, abnormal change intensity, and local region priority processing indicators of the local region are fused and recombined to determine the probability of local region defect occurrence and evolution trend. Based on the determination results, an aggregated defect score index for each local region is generated.

[0100] It should be noted that the transmittance anomaly in the polymerization defect scoring index is not only used to characterize internal structural anomalies, but also to reflect changes in moisture content, extract content, and related responses of effective components obtained from near-infrared online monitoring. When a local area simultaneously exhibits a continuous deviation in transmittance gradient, abnormal near-infrared absorption response, and uneven transmittance distribution in consecutive frames, and the changes correspond spatially to color or texture anomalies, the local area is identified as a high-interest area with both surface and internal quality response anomalies. This allows the polymerization defect scoring index to simultaneously serve the identification of appearance defects and the monitoring of the internal quality status of Chinese herbal medicine slices.

[0101] When determining the probability and evolution trend of defects in local areas, first read the multi-dimensional abnormal feature combination data of each local area, and compare the change magnitude, duration of frames and change direction of color abnormality, texture abnormality and light transmission abnormality in the current time series index.

[0102] When the same local area exhibits abnormal color enhancement, abnormal texture aggravation, or continuous deviation of the transmittance gradient in consecutive frames, and the potential defect development direction and change rate recorded in the defect evolution time-series feature vector remain continuously changing, the local area is determined to be a potential defect. Based on the persistence of abnormal features, the magnitude of abnormal changes, the stability of defect development direction, the change rate, and the local area priority processing index, the defect status of the local area is classified and organized to generate a corresponding aggregated defect scoring index.

[0103] When generating the aggregated defect scoring index, color anomalies, texture anomalies, light transmission anomalies, defect evolution time sequence characteristics, and local area priority processing indicators in the same local area are normalized to the range of 0 to 1. These features are then integrated and processed according to their contribution to defect determination to obtain the aggregated defect scoring index for the local area. The contribution of each feature is determined based on the differentiation effect between defective and non-defective samples in the results of manual review or processing quality inspection. The aggregated defect scoring index ranges from 0 to 1; a higher value indicates a higher probability of defects in the corresponding local area. When the aggregated defect scoring index is less than 0.40, it is classified as a low-defect-probability area; when it is greater than or equal to 0.40 and less than 0.70, it is classified as a suspected defect area; and when it is greater than or equal to 0.70, it is classified as a high-defect-probability area. This index is used as the basis for subsequent spatial coordinate mapping, generation of composite defect areas, and generation of a complete defect distribution map.

[0104] S5: Map the aggregated defect scoring index to the spatial coordinates of each Chinese herbal medicine slice, perform connectivity analysis and boundary fusion on adjacent and overlapping local areas to obtain composite defect regions, and perform spatial integration and coordinate normalization on all composite defect regions to generate a complete defect distribution map.

[0105] S5.1: The spatial coordinates of each piece of Chinese herbal medicine are mapped using the aggregation defect scoring index, and adjacent and overlapping local areas are extracted. Through boundary analysis and merging, integrated candidate defect areas are obtained.

[0106] Furthermore, the aggregated defect scoring index of each local area is bound to the corresponding spatial coordinates and regional boundary coordinates to form local area scoring coordinate data. Then, the local area scoring coordinate data is sorted according to the spatial coordinate position, and local areas with adjacent boundaries or overlapping coordinate ranges are extracted to form a set of local areas to be merged.

[0107] Based on the set of local regions to be merged, boundary continuity analysis is performed on the boundary coordinates of each local region. The internal boundaries in the overlapping parts are deleted, the outer boundaries of each local region are retained, and the endpoints of adjacent outer boundaries are connected to obtain continuous boundary contours. Based on the continuous boundary contours, the aggregated defect scoring index, spatial coordinates, and boundary ranges in the set of local regions to be merged are merged and organized to form integrated candidate defect regions.

[0108] S5.2: Based on the regional structure of the integrated candidate defect regions, spatial integration and coordinate normalization are performed on all integrated candidate defect regions, and they are uniformly mapped according to spatial location to obtain the composite defect region.

[0109] Furthermore, based on the integrated candidate defect regions obtained in the previous processing, the regional structure of each integrated candidate defect region is extracted according to the spatial coordinates of each piece of Chinese herbal medicine. The regional structure includes the boundary coordinate sequence, the set of internal connected pixels, the center position of the region, and the outer range of the region. Based on the center position of the region and the outer range of the region, the integrated candidate defect regions are spatially sorted, and integrated candidate defect regions belonging to the same piece of Chinese herbal medicine and having boundary contact, boundary intersection, or overlapping outer ranges are grouped into the same region group to be merged.

[0110] The region is filled and merged using the set of internally connected pixels within each region group to be merged. The boundary coordinate sequence is then filled with breakpoints, connected, and smoothed to obtain the outline of the merged region. Based on the spatial coordinates of each piece of Chinese herbal medicine, the outline, center position, and outer range of the merged region are uniformly processed to maintain the corresponding spatial relationship of the defective regions within the same piece of Chinese herbal medicine. Finally, the composite defective region is obtained by mapping and organizing the uniformly processed outline and the set of internally connected pixels.

[0111] S5.3: Based on the spatial coordinates of each local area mapped to the aggregated defect scoring index, the positions and boundaries of each composite defect area are summarized and aligned. By smoothing the boundaries of adjacent areas and adjusting their spatial positions, an integrated composite defect area data set is obtained.

[0112] Furthermore, based on the spatial coordinates of each local region mapped to the aggregated defect scoring index, the spatial coordinates, aggregated defect scoring index, and regional boundary points of each local region are read in the order of spatial coordinates of each piece of Chinese herbal medicine. Local regions with adjacent or overlapping relationships are taken as local regions to be summarized. Connectivity analysis is performed on the local regions to be summarized. According to the boundary contact relationship, boundary overlap relationship, and spatial continuity relationship between local regions, local regions belonging to the same defect range are merged into the corresponding composite defect region. The location data and boundary data of the corresponding composite defect region are summarized simultaneously.

[0113] Alignment processing is performed on the location and boundary data of the composite defect region. The spatial coordinates of each local area within the same composite defect region are adjusted to the same coordinate reference. Boundary fusion is performed on the boundary breakpoints, overlapping segments, and abrupt boundary segments of adjacent regions to form continuous boundaries between adjacent regions. After boundary fusion, the composite defect region is smoothed to remove boundary burrs, fill boundary breakpoints, and correct local positional deviations caused by image offset. The corrected region location, region boundary, and corresponding aggregated defect scoring index are then uniformly organized to obtain the integrated composite defect region data set.

[0114] S5.4: Based on the integrated composite defect region dataset, the location and boundary data of each composite defect region are normalized, the normalized composite defect regions are mapped to a spatial coordinate system, and the regional data are reorganized to generate a complete defect distribution map.

[0115] Furthermore, based on the integrated composite defect region dataset, the location data, boundary data, and aggregated defect scoring index of each composite defect region are extracted. The location data of each composite defect region is matched with the spatial coordinate range of the corresponding Chinese herbal medicine slice to obtain the relative location data of the composite defect region within each Chinese herbal medicine slice. Based on the relative location data, the boundary data of the composite defect region is scaled and oriented to be corrected, so that the location data and boundary data of each composite defect region are transformed to the same spatial coordinate scale, thus obtaining normalized composite defect region data.

[0116] Based on the normalized composite defect region data, the coordinate position, boundary range, and region order of each composite defect region are determined in the spatial coordinate system. The coordinate position, boundary range, and aggregated defect scoring index are then correlated and organized according to the spatial coordinates of the same Chinese herbal medicine slice to obtain spatially mapped region data. By removing overlaps, processing boundary continuity, and reorganizing the region order of the spatially mapped region data, a defect region arrangement result covering each Chinese herbal medicine slice is formed, and the defect region arrangement result is transformed into a complete defect distribution map.

[0117] It should be noted that the complete defect distribution map is generated by transforming normalized composite defect region data. The complete defect distribution map includes at least the spatial coordinates of each piece of Chinese herbal medicine, the boundary of the composite defect region, the location of the composite defect region, the corresponding aggregated defect scoring index, and the defect evolution direction. When generating the complete defect distribution map, all composite defect regions within the same piece of Chinese herbal medicine are arranged according to the normalized spatial coordinates, and the boundary of the composite defect region is associated with the corresponding aggregated defect scoring index, so that the complete defect distribution map can express the positional relationship, range relationship, and evolution relationship of the defect regions in each piece of Chinese herbal medicine.

[0118] In summary, this invention uses a defect evolution prediction model to continuously and dynamically model color changes, texture changes, and transmittance gradient changes, and forms a local area priority processing index based on the defect expansion rate. This allows the appearance abnormalities of Chinese herbal medicine slices to be associated with their internal light transmittance response in the same time-series chain, thereby improving the stability of identifying the development direction, rate of change, and range of potential defects.

[0119] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An online detection method based on machine vision and near-infrared fusion, characterized in that, include: The system continuously acquires visible light and near-infrared images of each piece of Chinese herbal medicine along the transport path, and dynamically adjusts the illumination angle of the light source according to the offset and blur caused by the transport motion to form vibration-compensated images. Pixel-level fusion of vibration-compensated images is performed, and the color distribution, texture features and light transmission features of each local region are analyzed to identify local microstructural differences and potential abnormal regions, generating multi-band feature fusion images. The multi-band feature fusion image is input into the defect evolution prediction model to dynamically model the color change, texture change and transmittance gradient change of the local area, identify the development direction, rate and evolution range of potential defects, form a defect evolution time series feature vector, and assign priority processing index to the local area according to the rate of defect expansion. Based on the time-series feature vector of defect evolution and the local region priority processing index, the color anomaly, texture anomaly and light transmission anomaly of the local region of the vibration compensation image and the multi-band feature fusion image are prioritized and multi-dimensionally fused and deduced to generate the aggregated defect scoring index of each local region. The aggregated defect scoring index is mapped to the spatial coordinates of each Chinese herbal medicine slice. Connectivity analysis and boundary fusion are performed on adjacent and overlapping local areas to obtain composite defect regions. All composite defect regions are then spatially integrated and their coordinates are normalized to generate a complete defect distribution map.

2. The online detection method based on machine vision and near-infrared fusion as described in claim 1, characterized in that, The specific steps for forming the vibration-compensated image are as follows: The surface color and internal structure information of each slice of medicine will be recorded by acquiring continuous visible light images and near-infrared images to obtain multimodal image data; Based on the image offset and blur caused by the conveying motion, the multimodal image data is dynamically aligned and reconstructed to extract the position information and local texture features of each pixel and obtain the vibration correction image after displacement compensation. By utilizing local feature changes in vibration-corrected images and dynamically adjusting the illumination angle of the light source, a vibration-compensated image is formed through illumination compensation and continuous frame fusion.

3. The online detection method based on machine vision and near-infrared fusion as described in claim 1, characterized in that, The specific steps for generating the multi-band feature fusion image are as follows: The color, texture and light transmission information of each pixel are obtained from the vibration-compensated image, and normalization is performed to eliminate the acquisition bias of the image to obtain local feature data. Pixel-level fusion of local feature data is performed. By analyzing the feature differences of each local region, color, texture and light transmission information are recombined in multiple bands to obtain a fused multi-band local feature map. Anomaly identification and microstructure difference assessment are performed on the fused multi-band local feature maps to generate a multi-band feature fusion image.

4. The online detection method based on machine vision and near-infrared fusion as described in claim 1, characterized in that, The specific construction process of the defect evolution prediction model is as follows: A defect evolution prediction model is constructed based on a multimodal local feature encoding layer, a defect evolution time series analysis layer, and a defect evolution feature generation layer. The multimodal local feature coding layer extracts, classifies, and normalizes the color, texture, and light transmission information of local regions in the multi-band feature fusion image to generate a set of local feature data. The defect evolution time-series analysis layer, based on multi-band feature fusion images, performs continuous frame sorting and time series indexing on local feature data sets, and generates local regional time-series evolution information through difference analysis, trend statistics and dynamic cumulative calculation. The defect evolution feature generation layer calculates a local region priority processing index based on the local region temporal evolution information, uses the local region priority processing index as an intermediate input, and performs multi-dimensional fusion and deduction with the local region temporal evolution information to generate a defect evolution temporal feature vector.

5. The online detection method based on machine vision and near-infrared fusion as described in claim 1, characterized in that, The specific steps for forming the defect evolution time-series feature vector are as follows: Based on multi-band feature fusion images, the color changes, texture features and light transmission features of each local region are extracted and classified to obtain a local feature data set, and a time series index is established by sorting consecutive frames; Based on the time series index, continuous frame comparison and trend analysis are performed on the local feature data set to generate local area color change trend, texture change trend and transmittance gradient change results, identify potential abnormal areas, and calculate the development direction and change rate of potential defects through cumulative analysis to form the temporal evolution information of the local area. By utilizing temporal evolution information, the rate of defect expansion in each local region is evaluated, a local region priority processing index is generated, and the priority processing index is combined with the color, texture and transmittance change trends to form a defect evolution temporal feature vector.

6. The online detection method based on machine vision and near-infrared fusion as described in claim 5, characterized in that, The process of performing continuous frame comparison and trend analysis on the local feature data set based on the time series index to generate local area color change trend, texture change trend and transmittance gradient change results, identify potential abnormal areas, and calculate the development direction and change rate of potential defects through cumulative analysis to form local area temporal evolution information includes performing difference analysis on the local feature data set of continuous frames, and identifying candidate areas with obvious abnormal change amplitudes by comparing the color, texture and transmittance changes of each local area. The candidate regions are evaluated for cumulative trends on the time series index, and the parts of the local region that show continuous abnormal changes in consecutive frames are identified as potential abnormal regions. Extract the dynamic evolution of potential abnormal regions, calculate the possible development direction and rate of change of defects, and form the results into time-series evolution information of local regions.

7. The online detection method based on machine vision and near-infrared fusion as described in claim 1, characterized in that, The specific steps for generating the aggregated defect scoring index for each local region are as follows: Based on the time-series feature vector of defect evolution and the local region priority processing index, color anomalies, texture anomalies and light transmission anomalies of each local region in the vibration compensation image and the multi-band feature fusion image are extracted, and each anomaly feature is normalized to obtain a local feature dataset. Based on the local feature dataset, the abnormal features of each local region are sorted according to the priority processing index, and the sorting results are fused and recombined with the temporal evolution information to generate multi-dimensional abnormal feature combination data for each local region. Based on multi-dimensional abnormal feature combination data, combined with local color abnormality, texture abnormality and light transmission abnormality information and defect evolution time sequence feature vector, it is determined whether there are defects in each local area, and an aggregated defect scoring index for each local area is generated.

8. The online detection method based on machine vision and near-infrared fusion as described in claim 7, characterized in that, The method of determining whether there are defects in each local area based on multi-dimensional abnormal feature combination data, combined with local color abnormality, texture abnormality and light transmission abnormality information and defect evolution time sequence feature vector, and generating aggregated defect scoring index for each local area includes analyzing the multi-dimensional abnormal feature combination data of each local area and identifying potential abnormal areas by comparing the change range of color abnormality, texture abnormality and light transmission abnormality. By combining the identified potential abnormal regions with the defect evolution time-series feature vector, the probability of defects occurring and the evolution trend in local regions are assessed, and an aggregate defect score index for each local region is generated based on the assessment results.

9. The online detection method based on machine vision and near-infrared fusion as described in claim 1, characterized in that, The process of mapping the aggregation defect scoring index to the spatial coordinates of each Chinese herbal medicine slice, performing connectivity analysis and boundary fusion on adjacent and overlapping local regions to obtain the composite defect region, is as follows: The spatial coordinates of each Chinese herbal medicine slice are mapped using the aggregation defect scoring index, and adjacent and overlapping local areas are extracted. Through boundary analysis and merging, integrated candidate defect areas are obtained. Based on the regional structure of the integrated candidate defect regions, all integrated candidate defect regions are spatially integrated and their coordinates are normalized, and then uniformly mapped according to their spatial location to obtain the composite defect region.

10. The online detection method based on machine vision and near-infrared fusion as described in claim 1, characterized in that, The specific steps for generating a complete defect distribution map are as follows: Based on the spatial coordinates of each local area mapped to the aggregated defect scoring index, the positions and boundaries of each composite defect area are summarized and aligned. By smoothing the boundaries of adjacent areas and adjusting their spatial positions, an integrated composite defect area dataset is obtained. Based on the integrated composite defect region dataset, the location and boundary data of each composite defect region are normalized, the normalized composite defect regions are mapped to a spatial coordinate system, and the regional data are reorganized to generate a complete defect distribution map.

Images