Winter jujube defect identification method and system based on deep learning

Abstract

The application discloses a kind of winter jujube defect identification method and system based on deep learning, it is related to agricultural automation sorting technical field, including the following steps: receiving the matrix data of multiple frame images of the same target winter jujube under different space irradiation angles Time acquisition;The brightness value and spatial gradient value of each coordinate element in the matrix data of each frame image are calculated, and the coordinate region with brightness value greater than the preset brightness threshold and spatial gradient value less than the preset gradient threshold is extracted as highlight blind area, and the corresponding highlight suppression mask matrix data is generated for the matrix data of each frame image.The application complements the effective features of multiple perspectives in feature dimension, successfully fuses and generates global skin feature matrix data representing the real physical skin state without reflection shielding, realizes 360-degree non-blind area information coverage on the surface of the same winter jujube, and greatly reduces the missed detection rate caused by highlight shielding.

Description

Technical Field

[0001] This invention relates to the field of agricultural automated sorting technology, specifically to a method and system for identifying defects in winter jujubes based on deep learning. Background Technology

[0002] In the field of automated sorting in modern agriculture, machine vision and deep neural network technologies have been widely applied to the surface quality inspection of spherical or ellipsoidal fruits and vegetables such as jujubes. In order to accurately screen out defective products, existing inspection systems typically deploy high-intensity industrial light sources and camera equipment on the production line. By acquiring two-dimensional images of the product surface and directly feeding them into an image processing model, the deep network architecture of the model is used to extract local textures and thus identify various types of surface damage.

[0003] To ensure that visual sensors can capture sufficiently clear surface details for algorithm analysis, providing the object under test with ample and high-intensity illumination is a current consensus prerequisite for obtaining high-quality detection data. However, for specific agricultural products like jujubes, which not only possess unique geometric curves but also naturally exhibit smooth, dense, and reflective skin properties, severe optical interference variables are inevitably introduced when these fruits with specular-like reflective properties are placed under concentrated illumination from a high-intensity light source. The interaction between the light source angle and the surface normal direction inevitably generates extremely bright, saturated light spots in specific areas of the fruit skin within the camera's field of view. With this overexposure, the original texture and color information of the affected area is completely masked, leading to severe distortion of the underlying pixels input into the deep learning model.

[0004] Under this optical feedback, the abrupt edges generated in the reflective area are easily mistakenly activated by the network as damage contours, causing normal fruits to be misjudged and eliminated. Furthermore, large-area light saturation can directly swallow up the real fine cracks or lesions hidden beneath, resulting in the missed detection of unqualified products. Conventional image feature calculation logic cannot distinguish the real boundary between light interference and physical damage. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for identifying defects in jujubes based on deep learning.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A deep learning-based method for identifying defects in jujubes includes the following steps:

[0008] Receive matrix data of multiple frames of images of the same target, winter jujube, acquired at different spatial illumination angles in a time-division manner;

[0009] Calculate the brightness value and spatial gradient value of each coordinate element in the matrix data of each frame image, and extract the coordinate regions with brightness values ​​greater than a preset brightness threshold and spatial gradient values ​​less than a preset gradient threshold as specular blind zones, and generate corresponding specular suppression mask matrix data for the matrix data of each frame image.

[0010] Based on the pre-stored convolution kernel weight matrix data, feature extraction is performed on the matrix data of each frame of image to obtain initial feature tensor data. The initial feature tensor data is then multiplied element-wise with the corresponding specular suppression mask matrix data to output specular de-suppressed feature tensor data.

[0011] The de-highlight feature tensor data output from different illumination angles are spliced ​​and recombined across dimensions to generate a global epidermal feature matrix data representing the absence of reflective occlusion.

[0012] The global epidermal feature matrix data and the preset classification mapping matrix data are subjected to a dimension reduction inner product operation to obtain the probability distribution vector data representing the defect category. When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output.

[0013] A deep learning-based defect identification system for winter jujubes includes:

[0014] The data acquisition module is used to receive matrix data of multiple frames of images of the same target, winter jujube, collected at different spatial illumination angles in a time-division manner;

[0015] The specular mask generation module is used to calculate the brightness value and spatial gradient value of each coordinate element in the matrix data of each frame image, and extract the coordinate regions with brightness values ​​greater than a preset brightness threshold and spatial gradient values ​​less than a preset gradient threshold as specular blind zones, and generate corresponding specular suppression mask matrix data for the matrix data of each frame image.

[0016] The feature extraction and suppression module is used to extract features from the matrix data of each frame of the image based on the pre-stored convolution kernel weight matrix data, obtain the initial feature tensor data, and perform element-wise multiplication operation between the initial feature tensor data and the corresponding specular suppression mask matrix data to output the specular de-spectral feature tensor data.

[0017] The cross-dimensional feature fusion module is used to perform cross-dimensional splicing and recombination operations on the de-highlight feature tensor data output under different illumination angles, and fuse them to generate global epidermal feature matrix data representing the absence of reflective occlusion.

[0018] The defect classification and control output module is used to perform a dimension reduction inner product operation on the global skin feature matrix data and the preset classification mapping matrix data to obtain the probability distribution vector data representing the defect category. When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] This invention overcomes the limitations of traditional methods that rely solely on pixel brightness to determine highlights by introducing spatial gradient values ​​as a joint constraint. Since physically reflective areas are typically extremely smooth (with minimal gradients), while real bright-colored defects (such as scratches and peeling) usually have rough edges and texture (with large gradients), this invention achieves accurate location and extraction of highlight blind spots by using a judgment logic where brightness exceeds a preset threshold and spatial gradient is less than a preset threshold. This effectively avoids the technical challenge of misjudging highlights as defects (false positives). Furthermore, it employs tensor-level (feature layer) highlight suppression mask calculation, avoiding artificial noise introduced by image-level processing and improving the model's feature extraction capabilities. A multi-view, cross-dimensional feature fusion architecture is constructed, enabling virtual reconstruction of the entire jujube skin without highlight blind spots, solving the problem of missed detections (false negatives). Addressing the physical limitation of inherent highlight blind spots in single-view images, this invention utilizes time-divisionally acquired images from different spatial illumination angles to extract the corresponding de-highlight feature tensors, which are then stitched and recombined along the channel depth dimension. Since the position of the highlight shifts spatially under different illumination angles, this algorithm complements and mosaics the effective features from multiple perspectives in the feature dimension, successfully fusing and generating a global epidermal feature matrix data that represents the real physical epidermal state without reflective occlusion. This achieves 360-degree blind-spot-free information coverage of the same jujube surface, significantly reducing the false negative rate caused by highlight occlusion. Attached Figure Description

[0021] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts. Wherein:

[0022] Figure 1 This is a diagram illustrating the method steps of the present invention;

[0023] Figure 2 This is a flowchart of the present invention. Detailed Implementation

[0024] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0025] Application Overview:

[0026] In the field of automated sorting in modern agriculture, especially in the defect identification of smooth, spherical fruits and vegetables (such as winter jujubes), the integrity of the skin texture features and the high signal-to-noise ratio are regarded as key indicators for measuring the accuracy of visual detection. The extraction of such high-quality features is essentially a process of non-destructive mapping of the physical morphology of the surface at the level of optical imaging and deep learning. That is, through uniform illumination projection and continuous acquisition from multiple perspectives, the visual sensor is used as an information capture carrier to accurately transmit the true reflectivity and topological changes of the fruit peel to the neural network, thereby reconstructing the semantic information of the skin with specific morphological features in the feature space.

[0027] However, existing technologies lack a decoupling and verification mechanism for optical interference between ambient lighting distortion and the true physical morphology of fruit peels. This results in an inability to accurately identify optical information oversaturation and feature mapping distortion problems in the forward propagation process of deep learning networks. Optical information oversaturation manifests as the system using high-frequency, multi-view acquisition but actually relying on a single-dimensional brightness threshold for image preprocessing. Feature mapping distortion manifests as a sudden drop in local pixel gradients due to specular reflection when the system extracts convolutional features, causing spatial discontinuities in the underlying feature map and the real texture being swallowed by highlights. Consequently, a strict semantic correspondence cannot be established between the physical reflection of photons and the activation of network weights, leading to misjudgments of false defects or fuzzy extraction of latent damage by the visual model. This, in turn, affects the accurate assessment of defect morphological features and the reliability of sorting decisions.

[0028] For example, in the jujube sorting line of an agricultural product processing plant, when the vision system performs high-speed image acquisition, conventional deep learning models can only capture bright reflective patches on the surface of the fruit peel, but cannot distinguish whether they are accompanied by hidden high-gradient real textures such as scratches and peeling. Furthermore, when the jujube has strong local reflection due to its spherical curved surface, the model only records the dead white appearance of oversaturated pixels and fails to detect the abnormal decay of effective activation points and the decrease in feature transfer efficiency during the continuous edge feature extraction process. Specifically, the system misjudges smooth highlight blind areas as white defects and gives a rejection signal, or misclassifies real cracks under highlight occlusion as normal lighting shot noise. As a result, the model continues to solidify the wrong spatial mapping pattern and cannot form a global feature matrix that conforms to the real physical surface state.

[0029] If the above problems are not solved, the defect identification system will continue to lose its ability to objectively judge the true state of the jujube skin. In particular, the failure to effectively identify and remove the oversaturation of optical information will cause the model to rely excessively on local brightness abrupt changes, resulting in the feature transmission path deviating from the principle of real texture topology, thereby weakening the continuity of defect edges. At the same time, the failure to correct the feature mapping distortion problem across dimensions will increase the information friction within the network's receptive field, making it impossible for the global skin features to present a complete form without reflective occlusion, ultimately causing the output defect probability distribution to lose its due confidence. As a result, the inaccuracy of the sorting control signal will systematically hinder the core requirement of the production line to achieve high-precision non-destructive screening, seriously affecting the achievement of automated quality control goals.

[0030] like Figure 1-2 As shown, a deep learning-based method for identifying defects in jujubes includes the following steps:

[0031] Step 1: Receive matrix data of multiple frames of images acquired time-division multiple times under different spatial illumination angles for the same target jujube. For the same target jujube at a specific inspection station on the production line, the system receives matrix data of multiple frames of images acquired time-division multiple times under different spatial illumination angles. By controlling the light source arrays distributed in different spatial locations to trigger in a microsecond-level time-division manner and synchronizing the exposure time of the camera equipment, the system can acquire independent reflection images of the target jujube skin from various directions. This data acquisition method ensures from the physical source that the highlight blind areas in each frame of the image present a staggered distribution in spatial coordinates, converting continuous optical signals into a multi-dimensional digital matrix, providing the underlying data source for subsequent high-dimensional feature complementation.

[0032] Step 2: Calculate the brightness and spatial gradient values ​​of each coordinate element in the matrix data of each frame image. Regions with brightness values ​​greater than a preset brightness threshold and spatial gradient values ​​less than a preset gradient threshold are extracted as specular blind spots. A corresponding specular suppression mask matrix is ​​generated for each frame image's matrix data. After acquiring the matrix data of each frame image, the system does not use a single image overexposure threshold for specular removal. Instead, it comprehensively calculates the brightness and spatial gradient values ​​of each coordinate element. The system strictly executes a dual physical judgment logic: only when the brightness value of a coordinate region is greater than a preset brightness threshold (indicating strong reflection) and its spatial gradient value is less than a preset gradient threshold (indicating smoothness and lack of texture within the region) is it determined to be a true specular blind spot caused by specular reflection. Based on this judgment, the system independently generates corresponding specular suppression mask matrix data for each frame image. This step effectively distinguishes between smooth optical reflections and rough bright color defects (such as scratches), avoiding the accidental deletion of real defect textures due to a one-size-fits-all brightness filtering.

[0033] Step 3: Based on the pre-stored convolutional kernel weight matrix data, feature extraction is performed on the matrix data of each frame of the image to obtain initial feature tensor data. The initial feature tensor data is then multiplied element-wise with the corresponding specular suppression mask matrix data to output de-spectrum feature tensor data. The system first calls the pre-stored convolutional kernel weight matrix data to perform a sliding traversal on the original image matrix data of each frame to extract features, constructing initial feature tensor data containing the global illumination response. Subsequently, the system maps the two-dimensional specular suppression mask matrix data generated in the previous step to a three-dimensional tensor space and performs an element-wise multiplication operation with the initial feature tensor data. Through this mathematical blocking at the deep feature map level of the network (rather than the input image level), the system forces the activation features corresponding to the specular blind areas to zero, outputting clean de-spectrum feature tensor data. While eliminating optical interference, it preserves the continuous spatial semantics of the unreflected areas of the original image to the maximum extent, preventing artificial edge noise introduced by image-level smearing modifications.

[0034] Step 4: The de-highlight feature tensor data output from different illumination angles undergo cross-dimensional splicing and recombination operations to generate a global epidermal feature matrix representing the absence of reflective occlusion. To compensate for the feature information gaps left by a single viewpoint after de-highlight processing, the system aligns and concatenates the de-highlight feature tensor data output from different illumination angles along the channel depth dimension. Since the highlight blind spots under different viewpoints have been physically offset, the system performs cross-dimensional feature recombination operations in the spliced ​​high-dimensional space, allowing the feature blind spots of a certain viewpoint to be filled by the effective features retained by other viewpoints under the same physical coordinates. After dimensionality reduction and fusion, the final global epidermal feature matrix representing the absence of reflective occlusion is generated. This matrix data completely reconstructs the true physical appearance of the target jujube from 360 degrees in the feature space, eliminating the information blind spots caused by a single light source.

[0035] Step 5: Perform a dimensionality reduction inner product operation on the global epidermal feature matrix data and the preset classification mapping matrix data to obtain probability distribution vector data representing the defect category. When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output. After obtaining the complete global epidermal features, the system reconstructs them and performs a dimensionality reduction inner product operation on them with the preset classification mapping matrix data, projecting the abstract spatial tensor features into values ​​corresponding to each defect category, and then transforming them into probability distribution vector data representing each category. To prevent the sorting mechanism from malfunctioning due to ambiguous features, the system introduces a double verification of the preset interception conditions: the system confirms the existence of a substantial defect only when the classification result with the highest probability falls within the target defect category range and its probability value clearly exceeds the preset high-confidence interception threshold, and then overwrites the status to the external physical execution port, outputting the corresponding sorting control signal data to remove the jujube; otherwise, the system remains silent and allows passage.

[0036] The core innovation of this invention lies in overcoming the limitations of traditional methods that rely solely on pixel brightness to determine highlights by introducing spatial gradient values ​​as a joint constraint. Since the interior of physically reflective areas is typically extremely smooth (with minimal gradient), while the edges of real bright-colored defects (such as scratches and peeling) are usually rough and textured (with large gradient), the invention achieves accurate localization and extraction of highlight blind spots by using a judgment logic where brightness exceeds a preset threshold and spatial gradient is less than a preset threshold. This effectively avoids the technical challenge of misjudging highlights as defects (false positives). Furthermore, the use of tensor-level (feature layer) highlight suppression mask calculation avoids artificial noise introduced by image-level processing, improving the model's feature extraction capabilities. A multi-view, cross-dimensional feature fusion architecture is constructed, enabling virtual reconstruction of the entire jujube skin without highlight blind spots, solving the problem of missed detections (false negatives). Addressing the physical limitation of inherent highlight blind spots in single-view images, the invention utilizes time-divisionally acquired images from different spatial illumination angles to extract the corresponding de-highlight feature tensors, which are then stitched and recombined along the channel depth dimension. Since the position of the highlight shifts spatially under different illumination angles, this algorithm complements and mosaics the effective features from multiple perspectives in the feature dimension, successfully fusing and generating a global epidermal feature matrix data that represents the real physical epidermal state without reflective occlusion. This achieves 360-degree blind-spot-free information coverage of the same jujube surface, significantly reducing the false negative rate caused by highlight occlusion.

[0037] Because the skin of winter jujubes is smooth and has mirror-like reflective properties, when using machine vision for defect identification, traditional methods of single-light source or simultaneous constant illumination from multiple light sources will result in large areas of light saturation (i.e., highlight areas) on a fixed region of the jujube surface. Within these highlight areas, the pixel brightness reaches the sensor's limit, masking the true texture and potential defect features of the jujube skin, preventing the visual inspection system from extracting effective information and leading to missed defects. Therefore, this paper proposes receiving matrix data of multiple frames of images of the same target winter jujube acquired at different spatial illumination angles over time, as follows:

[0038] Same target jujube: refers to a single jujube that is located at a specific physical inspection station of the machine vision system within the current inspection cycle.

[0039] Different spatial illumination angles: refers to the three-dimensional spatial distribution of industrial light source equipment relative to the center of the target jujube. It is usually defined by preset azimuth and pitch angle parameters to ensure that light is projected onto the surface of the jujube from different physical directions.

[0040] Time-sharing acquisition: refers to a hardware data acquisition method in which different light source devices are triggered sequentially and individually at set time intervals by a controller over a very short time axis, and the camera shutter exposure is controlled synchronously.

[0041] Multi-frame images: refers to a collection of static images containing time-series attributes generated from the same jujube tree within a complete time-division acquisition cycle.

[0042] Matrix data: refers to a two-dimensional or three-dimensional digital array structure containing specific values, formed by discretizing and quantizing continuous optical analog signals in an image according to spatial row and column arrangement and color channels.

[0043] Step 1: Calculation and Allocation of Light Source Trigger Timestamps: The system controller first calculates the trigger time points of light sources at various spatial angles based on the pipeline's operating speed to ensure that illumination signals do not overlap during time-division acquisition. The trigger time formula is:

[0044] ;

[0045] in, : indicates the first The absolute timestamp of when a spatial light source is lit and the image sensor begins to expose.

[0046] : Indicates the initial system time when the target jujube reaches the specified trigger position of the photoelectric sensor. : Indicates the sequence number of the currently triggered light source, and its value ranges from positive integer 1 to K (K is the total number of configured light sources, usually between 3 and 8). This represents the preset time interval constant between two adjacent image acquisitions. Considering the motion of the pipeline, to prevent significant physical displacement blurring of the images, The value range is typically set to 1 to 5 milliseconds. Industrial cameras require strict nanosecond-level synchronization with multiple light sources. Relying on arithmetic-level distributed timestamps, the control board can output precise pulse width modulation signals. This completely eliminates optical cross-interference caused by simultaneous illumination from different angles at the physical hardware level, ensuring that the optical reflection signal acquired by the system at each moment belongs to only a unique specific spatial angle.

[0047] Step 2: Matrix quantization of photoelectric signals:

[0048] At each timestamp The image sensor captures reflected photons at corresponding angles and converts them into digital matrix elements. The pixel quantization formula is: ;

[0049] in, : indicates the first At each illumination angle, the horizontal axis of the generated image matrix is... The vertical axis is Color channels are The numerical characteristic values ​​of the elements. For a typical eight-bit depth industrial camera, The value range of is truncated to the integer range of 0 to 255. : Represents the linear mapping and quantization function of the analog-to-digital converter inside the image sensor. : Indicates the sensor's position in physical coordinates during the exposure cycle. And specific wavelengths (color channels) The total energy of photons received and accumulated at the point of origin. The algorithm model cannot directly process continuous analog optical signals; it must rely on an analog-to-digital conversion function to convert photon energy into discrete characteristic values. This translates the complex physical optical reflection phenomenon into structured data arranged in fixed-step increments in computer memory, providing the basic computational units for subsequent matrix addition, subtraction, multiplication, and division operations.

[0050] Step 3: Construction and reception of multidimensional data sets: The quantized single-frame matrices are aggregated in memory. The dataset aggregation formula is: ;

[0051] in, : This represents the matrix data set that the system software ultimately receives, containing all the illumination angle features of the target jujube. : indicates the corresponding number The output of a single-frame 3D image matrix from each illumination angle consists of image width, height, and the number of color channels (e.g., red, green, and blue channels). To maintain the integrity and temporal consistency of data for the same target object under different lighting conditions, a contiguous buffer is allocated in the system memory to orderly package the discrete single-frame matrices, facilitating subsequent feature extraction modules to read the data in batches.

[0052] Example 1: Assume the production line is configured with Industrial light sources at different angles (light source 1, light source 2, light source 3). Set the time interval constant for time-division multiplexing. The initial time is 2 milliseconds. It is counted as 0 milliseconds.

[0053] Based on the timestamp allocation formula from the first step:

[0054] Trigger time of light source 1: millisecond.

[0055] Trigger time of light source 2: millisecond.

[0056] Trigger time of light source 3: millisecond.

[0057] The system acquired three frames of image matrix data. To demonstrate the specular displacement effect, a local image of the same physical location on the surface of the jujube was extracted. Pixel region (i.e., horizontal coordinate) Take values ​​from 1 to 3, with the vertical axis as the coordinate. Take 1 to 3), and only display the red single channel ( Pixel quantization value The data range is limited to 0 to 255.

[0058] correspond At time 1, receive matrix data fragments illuminated by light source 1. At this moment, the light source shines from the upper left, and the highlight area is concentrated in the upper left pixel:

[0059]

[0060] correspond At that moment, receive matrix data fragments illuminated by light source 2. At this moment, the light source shines from directly above, the highlight in the upper left corner fades, and the highlight area moves to the center:

[0061]

[0062] correspond At that moment, receive matrix data fragments illuminated by light source 3. At this moment, the light source shines from the lower right, the central highlight fades, and the highlight area moves to the lower right corner:

[0063]

[0064] Calculation and Result Receiving: The system executes the dataset aggregation formula:

[0065] .

[0066] The correlation comparison in the table above shows that, for this Any specific coordinate within the region (e.g., the center coordinate) , Although The matrix encountered a loss of highlight information with a value of 255, but in its received... Matrix (numerical value 160) and In the matrix (value 145), valid matrix data reflecting the normal physical properties of the coordinate point was obtained, thus completing the accurate data collection in this step.

[0067] In the aforementioned technology, multiple light source systems distributed in different spatial locations are employed. A time-division triggering strategy is executed by a controller, causing each light source to be lit individually and sequentially at preset minute time intervals. A synchronously controlled image sensor acquires images within the time window of each independent light source's illumination and reconstructs the photoelectric converted analog signals into digital matrix data with spatial coordinates and channel depth. Finally, multiple matrix datasets generated from the same jujube under different independent illumination directions are centrally received and stored. Because the illumination angle of the light source undergoes continuous spatial displacement along the time axis, the highlight area on the jujube surface also shifts position on the skin. After converting multiple frames of images into matrix data, for any spatial coordinate position representing the physical peel in the matrix, it will at most appear as highlight saturation data in the matrix for some illumination angles, while retaining the true peel pixel data unaffected by reflection interference in the matrix for other illumination angles. This breaks the physical limitation of fixed highlight blind zones at the data source level, providing underlying pixel data with spatially complementary characteristics, and providing the original data foundation for subsequent algorithms to perform highlight filtering and cross-dimensional feature stitching.

[0068] In visual inspection, highly reflective areas on the surface of jujubes appear as high-brightness features. However, genuine abrasions, peeling, or certain light-colored lesions on the jujube skin also exhibit high-brightness features. If the system uses only a single brightness threshold to filter highlights, it will inevitably misclassify these genuine bright defects as reflective areas and remove them, causing the deep learning model to lose crucial defect texture information, resulting in serious missed detections and misjudgments. Therefore, this paper proposes: calculating the brightness value and spatial gradient value of each coordinate element in the matrix data of each frame of the image, and extracting the coordinate regions with brightness values ​​greater than a preset brightness threshold and spatial gradient values ​​less than a preset gradient threshold as highlight blind zones, generating corresponding highlight suppression mask matrix data for the matrix data of each frame of the image, as detailed below:

[0069] Generate corresponding specular suppression mask matrix data for the matrix data of each frame of the image, specifically including:

[0070] Extract the pixel components of each coordinate element in the matrix data as brightness values;

[0071] Retrieve the preset two-dimensional discrete difference operator matrix, perform window traversal with a sliding step of one on the matrix data, calculate the pixel difference between the current center coordinate element and its horizontal and vertical neighbor coordinate elements respectively, aggregate the absolute values ​​of the differences in each direction, and output the spatial gradient value corresponding to the current center coordinate element.

[0072] When the brightness value of the coordinate element is greater than the preset brightness threshold and the spatial gradient value is less than the preset gradient threshold, the values ​​of the same relative coordinate addresses in the specular suppression mask matrix data are kept as the first weight constant used to block feature transmission.

[0073] When the brightness value of a coordinate element is less than or equal to a preset brightness threshold, or the spatial gradient value is greater than or equal to a preset gradient threshold, the value at the corresponding address is overwritten with a second weight constant used to maintain feature propagation.

[0074] Iterate through all coordinate elements in the matrix data and output the final specular suppression mask matrix data.

[0075] Pixel component: The numerical value that makes up the smallest physical unit of the image matrix. In a single-channel (grayscale) image, it represents the quantized value of light intensity; in a multi-channel image, it represents the light intensity data of a specific color channel (such as the red channel).

[0076] Spatial gradient value: A physical quantity that characterizes the degree of drastic change in pixel values ​​between a given coordinate point and its neighboring coordinate points in an image. The smaller the value, the smoother the region; the larger the value, the more likely the region has edges, texture, or a rough structure.

[0077] Two-dimensional discrete difference operator matrix: a mathematical analysis tool for digital image processing (equivalent to the derivative in a discrete state), which extracts the spatial rate of change of a pixel by performing addition and subtraction operations with specific weights on the center pixel and its surrounding pixels.

[0078] First weight constant: A preset control parameter used to block the forward propagation of features in deep learning networks. In this scheme, it is usually set to zero.

[0079] Second weight constant: A preset control parameter used to maintain the lossless transmission of original feature data in the network. In this scheme, it is usually set to a value of one.

[0080] Step 1: Extracting Brightness Values: The system iterates through the input single-frame image matrix data and extracts the pixel components of a specific channel as the brightness reference for that coordinate. The brightness extraction formula is:

[0081] ;

[0082] in, : indicates the first Frame image in spatial coordinates The brightness value at that location. : Continuing from the previous text, indicating the first In the image matrix at each illumination angle, the pixel quantization feature value corresponding to the red single channel (channel index set to 1) is used. Since the system requires a single scalar value to evaluate the overexposure degree of the target area, the complex image data is uniformly reduced to the brightness dimension that reflects light intensity, providing a direct judgment benchmark for subsequent threshold comparison.

[0083] Step 2: Calculate the spatial gradient value: The system retrieves a preset two-dimensional discrete difference operator, performs a window traversal with a sliding step size of one on the brightness matrix, calculates the absolute value of the brightness difference between the center coordinate and its four adjacent coordinates (up, down, left, and right), and sums them. The spatial gradient aggregation formula is:

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] in, : indicates the first Frame image in coordinates The spatial gradient value calculated at that location. , , , : Represents the center coordinates The brightness values ​​of adjacent coordinates on the left, right, top, and bottom sides. Since the pixel values ​​within the specular reflective areas of a jujube surface are typically highly consistent and smooth, while physical damage (such as cracks), even with high brightness, inevitably exhibits pixel steps at its local edges, this local smoothness is quantified using a first-order difference formula. This transforms the visible texture roughness into specific numerical values ​​that can be recognized by a computer, providing core topological data for decoupling highlights from bright color defects.

[0090] Step 3: Perform dual threshold determination and mask generation: The system retrieves the preset brightness threshold and preset gradient threshold from memory, performs joint logic determination for each coordinate point, and overwrites the mask matrix. The mask generation logic formula is as follows:

[0091] when and hour: ;

[0092] when or hour: ;

[0093] in, : indicates the first The two-dimensional specular suppression mask matrix data corresponding to the frame image in coordinates The mask value at that location. : Indicates the preset brightness threshold. For 8-bit images (values ​​from 0 to 255), this value is typically between 230 and 245 to extract overexposed areas. : Indicates the preset gradient threshold. Depending on the system camera's resolution and noise level, this value is typically between 150 and 230. : Represents the first weighting constant, with a value of 0. : Represents the second weighting constant, with a value of 1. Relying solely on a single brightness threshold can lead to real bright color defects being misidentified as highlights. Introducing logical AND and OR operations ensures that light spots are only blocked when both the conditions of "extreme brightness" and "smoothness" are met simultaneously. By generating a high-precision two-dimensional binary mapping table, accurate removal of optical artifacts is achieved while preserving the feature propagation path of real defects without loss.

[0094] Example 2: Continuing from the previous example Local matrix data fragments received at each time under illumination by light source 2 (At this time, let's assume) To calculate the boundary gradient, it is assumed that the brightness of the unknown pixels adjacent to the outer edge of the matrix is ​​all 240. At this point... Brightness matrix data As shown in the table below:

[0095]

[0096] Parameter value setting: Set the preset brightness threshold Set a preset gradient threshold .

[0097] First weighting constant Second weighting constant .

[0098] Calculation objective: Perform algorithm processing on the center overexposure point (2,2) and the edge overexposure point (3,2).

[0099] Calculation for coordinates (2,2):

[0100] Brightness extraction: .

[0101] Adjacent brightness: Left ;right ;superior ;Down .

[0102] Execute the spatial gradient aggregation formula:

[0103] .

[0104] Execution logic determination: brightness Satisfy; gradient Satisfies the condition. That is, the area is located at the center of a bright and relatively smooth light spot. Output mask: .

[0105] Calculation for coordinates (3,2): Luminance extraction: .

[0106] Adjacent brightness: Left Right margin hypothesis ;superior ;Down .

[0107] Execute the spatial gradient aggregation formula:

[0108] .

[0109] Execution logic determination: brightness Satisfies; but gradient This indicates that there is a severe texture break at this point (most likely a highlighted edge of a defect), and the condition that the spatial gradient is less than the preset threshold is not met. Output mask: .

[0110] The final generated specular suppression mask matrix data fragment The following table shows the results (all points are calculated by the system, and all points below the brightness threshold of 240 are output as 1):

[0111]

[0112] As can be seen from this example table and its derivation, the system accurately blocks the true smooth highlight center while preserving the bright edge features with high gradients.

[0113] The aforementioned technology employs a dual-constraint mechanism based on optical physical properties. In addition to calculating pixel brightness values, it simultaneously calculates two-dimensional spatial gradient values ​​reflecting local texture changes. By setting a joint logical condition where brightness exceeds a threshold and gradient is less than a threshold, a secondary screening of bright areas in the image is performed. Based on this joint judgment result, a two-dimensional mask matrix composed of blocking and preserving weights is constructed in memory. This mechanism can rigorously distinguish between smooth physical reflections and rough bright color defects based on physical properties. For genuine bright color defects, due to edge damage or abrupt texture changes, their spatial gradient must be greater than the preset gradient threshold, thus preventing them from being classified as highlight blind zones, and preserving their features. This dual-constraint mechanism accurately locates and isolates purely optical interference areas, generating highly targeted highlight suppression mask matrix data. This provides a high-precision physical barrier for subsequent deep learning networks in the feature extraction stage, improving the purity of defect recognition data.

[0114] In existing machine vision processing workflows, directly and crudely modifying pixel values ​​in highlight areas during the initial input image preprocessing stage—for example, by directly blacking out highlight areas or using surrounding pixels for numerical interpolation smoothing—directly disrupts the optical continuity and physical topology of the original image in two-dimensional space. When subsequent convolutional kernels sweep over the edges of these artificially modified regions during feature extraction, this strong pixel abrupt change causes the model to calculate false high-frequency gradient responses, introducing a large amount of artificial edge noise into the network. This not only interferes with the network's convergence to real damaged edges but also severely reduces the accuracy of sorting and recognition. Therefore, this paper proposes: Based on pre-stored convolutional kernel weight matrix data, feature extraction is performed on the matrix data of each frame of the image to obtain initial feature tensor data. The initial feature tensor data is then element-wise multiplied with the corresponding highlight suppression mask matrix data to output de-highlight feature tensor data, as detailed below:

[0115] Output specular de-spectral feature tensor data, specifically including:

[0116] The addressing window, which is matched to the size of the convolution kernel weight matrix data, performs a two-dimensional sliding traversal on the image matrix data;

[0117] The pixel data within each window is weighted and fused with the weight matrix, and the feature activation values ​​are output and cached sequentially to construct multi-channel three-dimensional initial feature tensor data.

[0118] The two-dimensional specular suppression mask matrix data is mapped to the various channel levels of the three-dimensional initial feature tensor data to establish a co-positional coordinate mapping relationship.

[0119] Traverse the memory physical addresses of the initial 3D feature tensor data, multiply the read feature activation values ​​with the corresponding coordinate mask weight constants, write the calculation results directly to the current memory address, and output the specular feature tensor data.

[0120] Convolution kernel weight matrix data: A set of parameters pre-trained offline with a large amount of data in a deep learning model, used as a two-dimensional or three-dimensional numerical array to extract specific edge, texture or morphological features in image sliding calculation.

[0121] Addressing window: The local receptive field region in the image matrix currently being processed during convolution operations, whose spatial size is strictly consistent with the physical size of the convolution kernel.

[0122] Feature activation value: A single scalar value obtained by multiplying and adding the inner product of the original pixel value and the convolution kernel weight matrix value within the addressing window, representing the response strength of the local region to a specific feature.

[0123] Initial feature tensor data: a multi-channel three-dimensional data structure formed by stacking all feature activation values ​​extracted from the entire image according to the original spatial coordinates and the newly generated channel depth, without any specular culling operation at this time.

[0124] Corresponding coordinate mapping relationship: Extend the mask matrix of the two-dimensional plane to the three-dimensional space to ensure that the horizontal and vertical coordinate points in the two-dimensional mask can be strictly aligned and act on all depth channels of the three-dimensional tensor under the same horizontal and vertical coordinates.

[0125] Step 1: Perform 2D sliding traversal and weighted fusion to extract features: The system retrieves the pre-stored convolutional kernel weight matrix data, controls the addressing window to slide row by row and column by column on the input image matrix of the k-th frame with a preset stride, and performs feature extraction calculations. The convolutional feature extraction formula is:

[0126] ;

[0127] in, : indicates the first Frame image in spatial coordinates At this point, the output channel is The feature activation values ​​in the initial feature tensor data. : Indicates the total number of channels in the input image matrix data (e.g., 3 for RGB images). : Indicates the index of the input color channel currently being calculated. : Indicates the side length of the preset convolution kernel addressing window, which is usually an odd number such as 3, 5 or 7. and : These represent the relative horizontal and vertical offset coordinate indices within the addressing window, respectively. : Continuing from the previous text, indicating the first The frame image matrix data corresponds to the coordinates and the actual pixel feature values ​​under the channel. : Represents the scalar weight value at the corresponding position in the pre-stored convolutional kernel weight matrix data. Its value is usually limited to between -0.1 and 0.1 during the model initialization phase and is fixed after training. : Indicates the index of the output feature channel currently being calculated, with a value ranging from 1 to 1. (The preset maximum number of output channels is usually set to 16, 32, or 64). Since the network needs to transform the underlying physical pixel data into abstract high-dimensional semantic features representing texture and damage contours, multiply-add inner products are a standard mathematical method for recognizing local texture responses. Multi-channel 3D initial feature tensor data containing all surface information (including defect features and specular interference features) is generated using a formula.

[0128] Step 2: Cross-dimensional mapping of mask matrix data: Obtain the two-dimensional specular suppression mask matrix data generated in the previous step, and broadcast and copy it along the depth channel direction in logical memory. The mapping transformation formula is:

[0129] ;

[0130] in, : indicates that after mapping to three-dimensional space, the corresponding number is... Frame image, coordinates ,aisle The three-dimensional mask weight constant. : Continuing from the previous text, indicating the first The two-dimensional specular suppression mask matrix data corresponding to the frame image in coordinates The constant value at (taken as the first weight constant) Second weighting constant The mask matrix is ​​a two-dimensional planar structure, while the initial feature tensor is a three-dimensional solid structure. Direct multiplication will cause matrix operations in the memory dimension to fail. By establishing a penetrating mapping relationship between the two-dimensional planar control signal and the three-dimensional depth feature channel, it is ensured that the specular determination of a coordinate point can synchronously control all dimensional features derived from that point.

[0131] Step 3: Element-by-element multiplication and feature interception:

[0132] Iterate through the physical memory addresses of the initial feature tensor data and multiply each address by its corresponding mask weight constant. The element-wise feature suppression formula is:

[0133] ;

[0134] in, : Indicates the final calculated output of the de-spectral feature tensor data in coordinates And the channel The feature values ​​below. Values ​​can be filtered most efficiently through address-based scalar multiplication. When the coordinates are in a specular blind zone, because... The mapping value is a blocking constant of zero, and the activation values ​​of all channel features corresponding to this region are forcibly reset to zero and overwritten; non-blind zone features are multiplied by a constant of one to achieve lossless preservation. The final output is clean de-highlight feature tensor data.

[0135] Example 3: Continuing from the previous example, for the corresponding The first time to obtain Frame image matrix data fragment and its corresponding red single-channel pixel value and the calculated output specular suppression mask matrix data fragment. Given the following data:

[0136] Center overexposure coordinates (2,2): pixel value mask value .

[0137] Edge overexposure coordinates (3,2): pixel value mask value .

[0138] Parameter settings: To avoid the data complexity introduced by boundary padding and to intuitively display feature operations, a preset convolution kernel size is set. (Right now (convolution kernel), number of input channels Set the index of a target output feature channel to be calculated. .

[0139] Set this Pre-stored weight constants of the convolution kernel .

[0140] Tensor operations for coordinates (2,2):

[0141] The first step is to calculate the initial features: because The addressing window only contains the current coordinates themselves (relative offset). , ):

[0142] .

[0143] The second step is mask mapping: extract the two-dimensional mask for this coordinate and map it to a three-dimensional mask weight.

[0144] .

[0145] Third step: Feature suppression: Perform element-wise multiplication. (The previously high-spectral-interference characteristic of 127.5 was successfully blocked and reduced to zero.)

[0146] Tensor operations for coordinates (3,2):

[0147] The first step is to calculate the initial features:

[0148] .

[0149] The second step is mask mapping: extract the two-dimensional mask for this coordinate and map it to a three-dimensional mask weight.

[0150] .

[0151] Third step: Feature suppression: Perform element-wise multiplication.

[0152] (Here, because the preceding steps determine the presence of high-gradient damage textures, the mask value remains constant, and the true defect features are fully preserved.)

[0153] The final output is the de-highlight feature tensor data. A local segment (for this) The corresponding table for channel levels is as follows:

[0154]

[0155] As can be seen from the above examples and tables, the system successfully outputs a feature array structure that removes ineffective light spots and retains real physical information in a specific channel dimension.

[0156] In the aforementioned technique, the filtering mechanism for highlight features is postponed to the deep feature extraction layer of the neural network. The system first maintains the continuity of the original image matrix data, using convolutional kernels to perform standard sliding window feature extraction, obtaining initial feature tensor data containing global information. Subsequently, on this multi-channel tensor dimension, an independent two-dimensional highlight suppression mask matrix is ​​introduced, and a cross-dimensional mapping relationship is established. Through a low-level element-wise multiplication mechanism, precise mathematical blocking of the activation response in highlight regions is implemented at the feature map level. By placing mask suppression at the feature level (tensor level), it ensures that the convolution operation is always based on the original continuous pixels with real physical illumination gradient properties, effectively eliminating the artificial edge noise problem caused by hard image modification. Simultaneously, based on the element-wise multiplication mathematical operation mechanism, it can both ensure the lossless transmission of the true semantic features of non-reflective areas along the channels and selectively intercept and clear erroneous interference features in highlight blind areas early in the network's forward propagation. This effectively improves the feature purity of the target tensor dataset, providing a high-quality data foundation for subsequent cross-view feature reconstruction of high-dimensional data.

[0157] Because after a single-view image undergoes specular suppression masking, although the feature interference caused by specular reflection is eliminated, the epidermal texture information originally located in the highlight area is also blocked and cleared, forming a data blind spot in the feature extraction space. If the single-view feature map with local data blind spots is directly input into the subsequent classification network, the network model will be unable to obtain the true physical state of the epidermis within the blind spot, leading to missed detection of potential defects distributed under the highlight area. To address this, we propose performing cross-dimensional concatenation and recombination operations on the de-spectrum feature tensor data output from different illumination angles to generate a global epidermal feature matrix representing the absence of reflective occlusion, as detailed below:

[0158] Generate global epidermal feature matrix data representing the absence of reflective occlusion, specifically including:

[0159] The high-dimensional cascaded tensor data is constructed by continuously arranging and splicing memory addresses along the channel depth dimension, including at least the de-highlight feature tensor data corresponding to the first illumination angle and the de-highlight feature tensor data corresponding to the second illumination angle.

[0160] Obtain scalar weight vector data with the same number of two-dimensional channel feature matrices as the high-dimensional cascaded tensor data. Use the scalar weight vector data to perform multiplication and normalization operations on each two-dimensional channel feature matrix in the high-dimensional cascaded tensor data to obtain the weighted tensor data after compensation and allocation.

[0161] Weighted summation and dimensionality reduction operations are performed on high-dimensional cascaded tensor data based on weighted tensor data to extract continuous edge features of areas not obscured by specular highlights, and output global epidermal feature matrix data.

[0162] De-highlight feature tensor data: This refers to the feature representation extracted from a single frame image at a specific illumination angle after suppressing highlight interference through a mask matrix in the pre-processing step. The value of this tensor at the original highlight locations is set to zero (blocked), while the true feature response is preserved at non-highlight locations.

[0163] High-dimensional cascaded tensor data: Physically stacking three-dimensional feature tensors from multiple illumination angles according to channel dimensions to form a unified data structure containing more channel depth.

[0164] Two-dimensional channel feature matrix: a logical slice inside the cascaded tensor, corresponding to the output of a specific feature detector (convolution kernel) under a specific illumination angle.

[0165] Scalar weight vector data: a set of one-dimensional numerical values, where each element corresponds to a two-dimensional channel feature matrix in a cascaded tensor, used to quantify the contribution or importance of that channel in the fusion process.

[0166] Step 1: Address concatenation and tensor concatenation along the channel dimension: The system retrieves the de-highlight feature tensor data corresponding to each illumination angle from the memory buffer and arranges them sequentially along the channel depth direction using physical memory addresses. The concatenation formula is:

[0167] ;

[0168] in: ;

[0169] : indicates the generated first High-dimensional cascaded tensor data under a global channel. : indicates the first The de-spectral feature tensor corresponding to each illumination angle. : Represents the spatial coordinate index of the characteristic matrix. : Illumination angle number, ranging from 1 to K. The total number of channels in the feature tensor at a single illumination angle. Channel index within a single illumination angle. To integrate multi-view features acquired over time into a single mathematical operation space while preserving spatial location correspondences, a feature pool containing full-view information was constructed, providing the physical basis for the system to retrieve effective features across angles.

[0170] Step 2: Compensation allocation and normalization based on scalar weights:

[0171] Obtain the scalar weight vector, perform weighted calculations on each channel matrix in the cascaded tensor, and perform normalization to ensure the balance of feature energies. The compensation allocation calculation formula is as follows:

[0172] ;

[0173] (in Accumulate from 1 to )

[0174] in, : Represents the weighted tensor data after compensation allocation. : Represents the scalar weight vector corresponding to the th The weight values ​​for each channel are typically between 0 and 1. The sum of weights is used as a normalization reference for subsequent dimensionality reduction. Since highlight removal at different angles can lead to the loss of features in some areas, it is necessary to adjust the influence of different channels on the final result through weights. This achieves the redistribution of feature energy among channels, strengthens the effective feature regions, and weakens the ineffective zero-value regions.

[0175] Step 3: Weighted summation, dimensionality reduction, and edge feature extraction:

[0176] The system performs accumulation on the weighted tensor data along the channel dimension, collapsing it into a two-dimensional matrix to extract continuous edge features. The weighted fusion dimensionality reduction formula is as follows:

[0177] (in Accumulate from 1 to );

[0178] in, : Represents the final output global epidermal feature matrix data. Dimensionality reduction is a necessary means to transform high-dimensional abstract features into physical texture representations that can be used for classification. Through summation, the "zero-value features (highlight blind spots)" at a certain viewpoint are compensated by the "non-zero effective features" at that location from other viewpoints. The final output matrix eliminates the interference of reflective spots, presenting a continuous and complete jujube epidermal texture.

[0179] Example 4: Continuing from the previous example, for the same position (2,2), assume that the system has feature data of two illumination angles.

[0180] Initial specular de-highlighting feature data acquisition: Angle 1 at (2,2) is a specular highlight (suppressed): Feature value .

[0181] Angle 2 at (2,2) is within the normal region: eigenvalue .

[0182] Performing the first step of concatenation: The two channel values ​​of the concatenated tensor at coordinate (2,2) are:

[0183] ;

[0184] .

[0185] Perform the second step of weighted compensation: Set the scalar weight vector as... .

[0186] Channel 1 weighted value: .

[0187] Channel 2 weighted value: .

[0188] Weighted sum .

[0189] Perform the third step: weighted summation and dimensionality reduction.

[0190] .

[0191] Feature fusion mapping table:

[0192]

[0193] As can be seen from the calculation, the epidermal feature information that was originally lost in angle 1 was effectively recovered and reconstructed in the final global feature matrix.

[0194] Since high-dimensional cascaded tensor data is composed of features stitched together from different illumination angles, and due to specular suppression in the pre-processing step, some channels may contain large areas of zero-value mask blind zones (extremely low information density), while other channels contain complete defect textures (extremely high information density). If these channels are treated equally during subsequent fusion (i.e., using a fixed average weight), the effective information will be diluted by the zero values ​​of the ineffective information, weakening the expression strength of the defect features. Therefore, this paper proposes that scalar weight vector data be dynamically generated by evaluating the feature distribution of the high-dimensional cascaded tensor data, specifically including:

[0195] Traverse each two-dimensional channel feature matrix in the high-dimensional cascaded tensor data, and calculate the feature mean data of all coordinate elements in each two-dimensional channel feature matrix;

[0196] The feature mean data of all two-dimensional channel feature matrices are arranged in the order of the physical arrangement of the channels and combined to generate a one-dimensional vector data of global illumination response that represents the current global illumination response state of winter jujube.

[0197] Perform matrix multiplication between the one-dimensional vector data of the global illumination response and the preset first dimension reduction mapping matrix data to generate the initial inner product vector data;

[0198] Iterate through each component value in the initial inner product vector data. When the component value is less than zero, overwrite the value in the corresponding memory address with zero. When the component value is greater than or equal to zero, keep the original value unchanged and output the transition vector data after non-linear truncation.

[0199] Perform matrix multiplication between the transition vector data and the preset second-dimensional mapping matrix data to generate reconstructed vector data;

[0200] For each component value in the reconstructed vector data, the power calculation is performed with the natural constant as the base and the opposite of the component value as the exponent. After adding the constant one to the calculation result, the reciprocal operation is performed to strictly converge all component values ​​to the value range of zero to one. Finally, a dynamic scalar weighted vector data with the same number of elements as the number of channels is output.

[0201] Feature mean data: The average value obtained by summing the feature activation values ​​of all spatial coordinate points in the same two-dimensional channel and dividing by the total number of coordinate points. This value physically represents the overall response energy or global feature activity of the channel under the current illumination angle.

[0202] Global illumination response one-dimensional vector data: A one-dimensional sequence formed by concatenating the feature mean data of each extracted channel in channel order, used to describe the feature energy distribution of the whole jujube under different illumination conditions at the macro level.

[0203] The first dimensionality reduction mapping matrix data is a set of weight parameter matrices pre-trained and fixed in the deep learning model. The number of rows is equal to the total number of channels, and the number of columns is less than the total number of channels. It is used to compress high-dimensional channel response information into a low-dimensional hidden layer space in order to extract the dependency correlation features between channels.

[0204] Nonlinear truncation: An activation function operation (i.e., rectified linear unit operation) that breaks the pure linear superposition limitation of matrix multiplication by forcing all negative components to zero, enabling the network to learn complex nonlinear mapping relationships.

[0205] The second upscaling mapping matrix data: another set of pre-trained weight parameter matrices, whose number of rows is equal to the dimension of the hidden layer after dimensionality reduction, and whose number of columns is restored to the total number of channels, is used to reproject the compressed and extracted associated features back into the original channel dimension space.

[0206] Step 1: Calculate the feature mean data for each channel: The system iterates through each depth channel in the high-dimensional cascaded tensor data, extracts the corresponding two-dimensional channel feature matrix, and calculates the arithmetic mean of all coordinate elements within it. The formula for calculating the feature mean is:

[0207] ;

[0208] in, : indicates the first The feature mean data is calculated from the feature matrix of each two-dimensional channel. : Represents the vertical physical height (number of rows) of the two-dimensional channel feature matrix. : Represents the horizontal physical width (number of columns) of the two-dimensional channel feature matrix. : These represent the horizontal and vertical spatial coordinate indices, respectively. : Continuing from the previous text, this indicates that high-dimensional cascaded tensor data is in coordinates And the channel index is The characteristic values ​​at the location. : Represents the global channel index, with values ​​ranging from 1 to the preset total number of channels. The system needs to eliminate interference from spatial location details and only obtain the total measure of each independent channel for the current feature extraction, which serves as the basis for evaluating the importance of that channel. By compressing the 3D tensor data containing rich spatial structure into single-value representations that retain only the channel dimension, the amount of data required for subsequent computations is greatly reduced.

[0209] Step 2: Generate one-dimensional vector data: The system arranges the calculated feature mean data in one dimension according to the order in which the channel indices in the memory address are added. The vector combination formula is:

[0210] ;

[0211] in, : Represents the combined one-dimensional vector data of the global illumination response, whose length is equal to the total number of channels. To meet the input format requirements of subsequent matrix multiplication, a state vector that can macroscopically reflect the information density of all current channels is constructed, and the discrete scalar values ​​are structured into a standard one-dimensional array.

[0212] Step 3: Perform the first dimension-reduced mapping matrix multiplication: The system retrieves the pre-stored first dimension-reduced mapping matrix data and performs an inner product operation with the one-dimensional global illumination response vector data. The initial inner product calculation formula is:

[0213] ;

[0214] in, : Represents the first element in the generated initial inner product vector data. Each component value. : indicates the first dimension reduction mapping matrix data. Line number The weight value of the column. : Represents the index of the hidden layer components after dimensionality reduction, with values ​​ranging from 1 to the preset reduced dimension R (R is strictly less than Z). Calculating weights directly in the original dimension would introduce an excessive number of parameters. Dimensionality reduction forces the network to learn the correlation mechanism between channels, removing redundant information from channel features and generating initial inner product vector data that condenses the interaction information between channels.

[0215] Step 4: Perform nonlinear truncation operation: The system sequentially accesses the memory address of each component in the initial inner product vector data and determines the sign of its value. The nonlinear truncation formula is:

[0216] ;

[0217] in, : Represents the first digit in the output transition vector data after nonlinear truncation. Each component value. : This indicates the operation of taking the maximum of two values. The matrix multiplication described above is merely a linear transformation of spatial coordinates. Introducing a truncation operation that returns negative values ​​to zero can inject nonlinear discriminative ability into the model. It filters out negative response features that may cause inverse interference, selecting transition vector components with positive activation significance.

[0218] Step 5: Perform the second-dimensional mapping matrix multiplication: The system performs an inner product calculation on the transition vector data and the pre-stored second-dimensional mapping matrix data, restoring the data dimension to the original number of channels. The reconstructed inner product calculation formula is:

[0219] ;

[0220] in, : Represents the first in the generated reconstructed vector data Each component value. : Represents the second dimension of the upgraded mapping matrix data. Line number The column weights. Each channel in the high-dimensional cascaded tensor data needs to be assigned a corresponding weight, therefore the hidden layer data must be upgraded to a higher dimension. This formula generates a set of original score components that correspond one-to-one with the original channels, i.e., the reconstructed vector data.

[0221] Step 6: Perform power and reciprocal operations on the natural constant: The system iterates through and reconstructs the vector data, performing exponential normalization calculations on each component. The dynamic scalar weight generation formula is:

[0222] ;

[0223] in, : Represents the dynamic scalar weight vector data of the final output corresponding to the first... The weighted components of each channel. : Represents the natural constant, approximately equal to 2.71828. Since the numerical range of the components in the reconstructed vector data is uncertain, directly multiplying them by the original feature tensor will cause feature values ​​to overflow. It is necessary to smoothly and rigorously compress them to a standard scaling factor range. All weight components are mapped to convergence within the (0,1) value range. The closer the value is to one, the more effective the channel feature is (e.g., not obscured by specular highlights); the closer the value is to zero, the greater the interference or the more severe the information loss in the channel feature.

[0224] A dynamic feature distribution evaluation mechanism was inserted before channel fusion. Through a mathematical process involving mean extraction, dimensionality reduction compression, nonlinear activation, dimensionality upscaling, and sigmoid normalization, a set of scalar weights corresponding to each channel was dynamically derived based on the real-time state of the input features. This mechanism adaptively quantifies the contribution of each viewpoint channel, automatically assigning high weights to channels with complete information and low weights to channels severely blocked by highlights, resulting in information loss. This dynamic compensation mechanism makes the final feature reconstruction more inclined towards high signal-to-noise ratio regions, improving the model's robustness in complex reflective environments.

[0225] Example 5: Continuing from the previous example, assume the system receives a spliced ​​tensor from two illumination angles, therefore the total number of channels is... Set the physical height of this tensor space. ,width .

[0226] Parameter value setting: Assume the dimension after dimensionality reduction is set. .

[0227] Preset first dimensionality reduction mapping matrix data The value is: , .

[0228] Preset second-dimensional mapping matrix data The value is: , .

[0229] Assuming by traversal The matrix is ​​used to calculate the eigenvalue of channel 1. The characteristic mean of channel 2 .

[0230] Combine to generate a one-dimensional vector data of global illumination response: .

[0231] Calculate the dimension according to the formula. Initial inner product on:

[0232] .

[0233] Since the example needs to demonstrate non-linear truncation and effective activation, we will modify it. The parameters are set as follows: .

[0234] Recalculate: .

[0235] judge The value 1.3 is greater than zero.

[0236] Transition vector data: .

[0237] According to the formula, Restore to Dimension.

[0238] Component 1: .

[0239] Component 2: .

[0240] Reconstructing vector data: .

[0241] Calculate the scalar weights for each channel:

[0242] Channel 1 weight: .

[0243] Channel 2 weights: .

[0244] Result mapping table:

[0245]

[0246] Through the above deduction steps, the system finally outputs scalar weight vector data [0.739, 0.372] with the same number of elements as the number of channels.

[0247] The high-dimensional cascaded tensor fuses de-highlight features from different angles. Due to variations in illumination intensity and masking blockage at different angles, simple concatenation leads to highly uneven energy distribution within the feature space. Directly transmitting these features with vastly different response intensities to subsequent classification networks can cause network training to fail to converge, or even trigger numerical explosions, severely impacting defect recognition accuracy. Therefore, this paper proposes using scalar weight vector data to perform product and normalization operations on the feature matrices of each two-dimensional channel in the high-dimensional cascaded tensor data, obtaining a compensated weighted tensor data, specifically including:

[0248] Analyze the dynamic scalar weight vector data and extract the scalar weight component values ​​corresponding to the feature matrix of each two-dimensional channel;

[0249] For high-dimensional cascaded tensor data, we traverse each two-dimensional channel feature matrix within it, and perform point-by-point multiplication operations on the corresponding scalar weight component values ​​with the pixel feature values ​​at each coordinate position in the matrix, outputting primary weighted tensor data.

[0250] Access the memory address where the dynamic scalar weight vector data is stored, sum all the component values ​​contained therein, and obtain the weight normalization factor data that represents the current global feature energy gain.

[0251] Iterate through and calculate the ratio of each element value in the primary weighted tensor data to the weight normalization factor data, overwrite the calculation results to the preset memory buffer, and output the final compensated weighted tensor data.

[0252] Analysis: This refers to the process of extracting the numerical components that correspond one-to-one with the target channel from continuously stored vector data according to specific data offsets and indexing logic.

[0253] Primary weighted tensor data: refers to the intermediate state feature data of high-dimensional cascaded tensors after multiplication by channel weights, before global amplitude scaling and normalization.

[0254] Weight normalization factor data: refers to the sum of all weight components involved in the current feature allocation, representing the total gain of global feature energy.

[0255] Memory buffer: A physical storage space with a fixed address range pre-allocated by the system to temporarily store intermediate variables or final results during the calculation process, in order to improve the processor's data read and write efficiency.

[0256] Step 1: Parsing and Extraction of Scalar Weight Vector: The system accesses the starting address of the dynamic scalar weight vector through addressing instructions and extracts the scalar weight components corresponding to each channel according to the channel index order of the high-dimensional cascaded tensor. The parsing and extraction correspondence is as follows:

[0257] ;

[0258] in, : indicates that the extracted value corresponds to the first The scalar weight component values ​​of a two-dimensional channel feature matrix. : Represents scalar weight vector data stored in memory. : Represents the index of the currently processed channel. The scalar weight vector is stored in one-dimensional form, while feature extraction is performed in three-dimensional tensor space, requiring an explicit mapping relationship. This relationship achieves spatial alignment between weight information and feature channels, laying the parameter foundation for subsequent pointwise operations.

[0259] Step 2: Generate the primary weighted tensor through pointwise multiplication:

[0260] The system iterates through each element within the high-dimensional cascaded tensor, performing point-to-point multiplication with the weight components of its respective channel. The product formula is:

[0261] ;

[0262] : Indicates the primary weighted tensor data in coordinates And the channel index is The eigenvalue at that location. : Represents the feature value at the corresponding position of the input high-dimensional cascaded tensor data. : Represents the scalar weight component corresponding to this channel. Through weight multiplication, the contribution of channels from different perspectives can be adjusted differentially, strengthening effective features and suppressing ineffective features. This initially alters the response intensity of different channels, forming a primary weighted tensor reflecting differences in importance.

[0263] Step 3: Calculate the weight normalization factor:

[0264] The system uses an adder to sum the values ​​of all components in the scalar weight vector to obtain the total global energy. The normalization factor is calculated using the following formula:

[0265] ;

[0266] in, : Represents the weighted normalization factor data that characterizes the current global feature energy gain. : Represents the total number of channels in a high-dimensional cascaded tensor. : indicates the first The scalar weights of each channel are used. The sum of the weight components determines the amplification of the feature energy. Without normalization, the eigenvalues ​​may overflow as the weight values ​​increase. Normalization yields a scalar value that represents the global gain baseline.

[0267] Step 4: Perform normalization ratio calculation and overwrite the result: The system iterates through the primary weighted tensor data, calculates the quotient of each element with the normalization factor, and writes the result to the memory buffer. The normalization ratio formula is:

[0268] ;

[0269] in, : Represents the weighted tensor data after compensation allocation in the final output. : Represents the weighted normalization factor data. The weighted feature values ​​are scaled to a stable numerical distribution range through division, ensuring consistency of feature energy across different samples. The output is a numerically stable, reasonably weighted final weighted tensor, eliminating imbalances in feature amplitudes across different perspectives.

[0270] This is achieved by introducing dynamic parsing, pointwise multiplication, and a normalization mechanism based on the sum of global weights. This is done by performing [the process] on each channel. The equivalent weighting coefficients were processed to achieve adaptive compensation and allocation of feature energy. This ensures that the fused feature tensor not only possesses complementarity between viewpoints but also numerical stability and scientific validity. Normalization effectively prevents excessive enhancement or attenuation of feature responses, enhancing the model's robustness to complex lighting environments and providing high-quality data input for subsequent accurate identification of subtle defects.

[0271] Example 6: Continuing from the previous example, assume a high-dimensional cascaded tensor It has 2 channels, each channel has a size of Analyzing dynamic weighted data:

[0272] scalar weight vector .

[0273] Parsing and extraction: .

[0274] Generate primary weighted tensor data: Assume the top-left corner feature of the first channel The corresponding position of the second channel .calculate:

[0275] ;

[0276] ;

[0277] Calculate the weight normalization factor:

[0278] ;

[0279] Normalize and output the final result:

[0280] Normalize the first channel: ;

[0281] Normalize the second channel: ;

[0282] Calculation result mapping table:

[0283]

[0284] Through this process, channel 1, which originally had higher energy, was moderately suppressed, while channel 2, which originally had lower energy, was relatively compensated, thus achieving a dynamic balance of global feature energy.

[0285] While direct superposition of feature tensors from multiple perspectives for dimensionality reduction solves the information loss problem caused by high-light blind spots, the superposition operation also introduces a new problem: a decrease in signal-to-noise ratio. Illumination shot noise and normal minor texture fluctuations in the fruit peel from different perspectives are amplified and accumulated during the summation process. If this two-dimensional feature matrix containing a large amount of high-frequency background noise is directly fed into the subsequent classification mapping layer, the classifier is easily interfered with and may misclassify the accumulated noise as epidermal abrasions or minor lesions, leading to an increase in the false positive rate. To address this, we propose performing weighted summation dimensionality reduction on high-dimensional cascaded tensor data based on weighted tensor data, extracting continuous edge features in areas not obscured by high light, and outputting global epidermal feature matrix data, specifically including:

[0286] For the weighted tensor data after compensation and allocation, the spatial coordinates of the two-dimensional plane are used as the reference index for traversal, and multiple weighted feature component data corresponding to the same spatial coordinate position at all channel depths are extracted.

[0287] The weighted feature component data extracted at the same spatial coordinate position are summed and the unique aggregated feature value corresponding to that spatial coordinate position is output. By traversing all spatial coordinate positions, the initial aggregated matrix data is obtained by compressing the dimension of the three-dimensional data structure into two dimensions.

[0288] Get the preset edge response threshold, iterate through all elements in the initial aggregation matrix data, and determine whether each element is less than the edge response threshold.

[0289] If so, it is determined to be smooth background or lighting shot noise, and the memory value of the corresponding element is multiplied by a preset attenuation coefficient for suppression.

[0290] If not, it is determined to be a real defect edge activation point, and its memory value remains unchanged;

[0291] The output is a global epidermal feature matrix data that enhances the response of effective edge activation points.

[0292] Weighted feature component data: refers to the discrete feature values ​​located at the same two-dimensional spatial coordinate point and distributed along the depth direction in various specific channels after the high-dimensional cascaded tensor data has been compensated by scalar weight product.

[0293] Initial aggregate matrix data: This refers to the two-dimensional transition matrix generated by summing all components of the three-dimensional weighted tensor data along the channel depth direction. This matrix has not yet been filtered for background noise.

[0294] Edge response threshold: A pre-set threshold value in the system used to distinguish whether the characteristics of the spatial coordinate point are caused by actual physical damage to the surface of the jujube (such as cracks or dark spots, which are characterized by high response) or by smooth fruit skin or light refraction (characterized by low response).

[0295] Illumination shot noise: In the process of image acquisition and feature extraction, small, discrete, non-real feature activation signals caused by random fluctuations of photons from the light source or dark current of the sensor are usually represented as isolated points with low values ​​in the matrix.

[0296] Attenuation coefficient: A preset scaling factor (usually greater than zero and less than one) is used to proportionally reduce background features below the response threshold to achieve numerical suppression rather than forced zeroing, thereby preserving basic topological continuity.

[0297] Step 1: Spatial Coordinate-Based Traversal and Weighted Feature Component Extraction: The system uses the horizontal and vertical spatial coordinate system of a two-dimensional plane as the reference index, traversing the compensated and weighted tensor data output from the previous step row by row and column by column. At each specific spatial coordinate position, the system extracts multiple weighted feature component data corresponding to that position across all channels along the channel depth direction of the tensor.

[0298] Step 2: Summation of corresponding features and dimension compression reconstruction

[0299] For multiple weighted feature component data extracted at the same spatial coordinate location, the system uses an adder to perform an accumulation operation, compressing and reconstructing the three-dimensional data structure into two-dimensional data, and outputting the initial aggregated matrix data. The summation and dimensionality reduction calculation formula is as follows:

[0300] ;

[0301] in, : Represents the coordinates of the calculated initial aggregation matrix data in two-dimensional space. The unique aggregated feature value at that location. : Represents the weighted tensor data after compensation allocation in spatial coordinates And the channel index is Weighted feature component data at the location. : Indicates the index variable in the channel depth direction. : Represents the total number of channels in the weighted tensor data. After specular masking and weighted compensation, the effective features from different viewpoints are distributed across the channels. By summing along the depth dimension, the effective feature responses from different viewpoints at the same physical location can be superimposed. This effectively eliminates the independence of viewpoints, flattens and integrates multi-dimensional information representing the same physical region into a single two-dimensional plane, forming an initial epidermal feature map with a global perspective.

[0302] Step 3: Threshold Determination and Shot Noise Attenuation Suppression: The system obtains the preset edge response threshold and attenuation coefficient, traverses all elements in the initial aggregation matrix data, and performs conditional judgment and numerical overwriting on each element. The suppression processing logic and formula are as follows:

[0303] when hour: ;

[0304] when hour: ;

[0305] in, : Indicates the final output global epidermal feature matrix data in coordinates The characteristic values ​​at the location. : Represents the preset edge response threshold. Depending on the activation strength of the feature extraction network in actual deployment, this value is usually set in the range of 10.0 to 50.0. : Represents the preset attenuation coefficient. To suppress but not completely destroy gradient continuity, this value is usually set between 0.1 and 0.3. Simple summation operations, while superimposing effective signals, inevitably also superimpose background noise and weak interference in each channel. Setting a hard threshold in conjunction with the attenuation coefficient can artificially widen the numerical difference between smooth backgrounds and real defects. For smooth backgrounds or noise below the threshold (determined as non-defect areas), their values ​​are further compressed; for defect edge activation points above or equal to the threshold, their values ​​are fully preserved. This significantly improves the contrast and signal-to-noise ratio of the feature matrix.

[0306] After performing summation and dimensionality reduction, a nonlinear suppression step is added. A conditional judgment mechanism based on edge response thresholds is introduced to explicitly divide the aggregated feature responses into effective edge regions and background noise regions. For the background noise region, a soft suppression is performed by multiplying by an attenuation coefficient less than one, rather than simply setting it to zero. This not only completes the feature collapse from a three-dimensional high-dimensional space to a two-dimensional plane, reconstructing continuous physical epidermal information without reflective occlusion, but also enhances and highlights the edge contours of effective defects, while ineffective background clutter is deeply suppressed. The output global epidermal feature matrix data has extremely high feature purity, significantly reducing computational interference to the backend classification model.

[0307] Example 7: Continuing from the previous example, assume the weighted tensor data after compensation allocation. Include There are 1 channel. We perform aggregation and suppression operations at coordinates (1,1) and (1,2) respectively.

[0308] The data at (1,1) calculated previously is as follows: , .

[0309] Here, we introduce the low-response feature data of adjacent coordinates (1,2) as follows: , Parameter value settings:

[0310] Set a preset edge response threshold .

[0311] Set the preset attenuation coefficient .

[0312] Calculation for spatial coordinates (1,1):

[0313] Apply the summation and dimensionality reduction formula:

[0314] .

[0315] Execution threshold determination and suppression processing:

[0316] judge numerical value Is it less than .

[0317] The result is negative, therefore it is determined to be a genuine defect edge activation point.

[0318] Keep the values ​​unchanged: .

[0319] Calculation for spatial coordinates (1,2):

[0320] Apply the summation and dimensionality reduction formula:

[0321] .

[0322] Execution threshold determination and suppression processing:

[0323] judge Is the value 20 less than .

[0324] The result is yes, and it is determined to be either smooth background or illumination shot noise.

[0325] Apply the attenuation formula: .

[0326] Feature dimensionality reduction and suppression relationship mapping table:

[0327]

[0328] As can be seen from the above example calculation, the effective region with high response (value 70) is fully preserved, while the background noise region with low response (value decays from 20 to 4) is effectively suppressed, resulting in a significant optimization of the signal-to-noise ratio of the output feature matrix.

[0329] The aforementioned technology fully utilizes the physical characteristics of jujubes under multi-source time-division illumination, specifically the physical position shift of the highlight blind zone in images from different viewing angles. At the feature level, the system first concatenates and stitches multiple de-highlight tensor data containing different blind zone locations along the depth channel; then, it uses extracted weight values ​​to multiply and compensate the data of each channel to balance the feature energy loss caused by the blocking operation; finally, the system performs a weighted summation operation across the channel dimension, reconstructing the deep high-dimensional tensor into a two-dimensional single feature matrix. Through cross-dimensional stitching and dimensionality reduction operations, the system achieves cross-spatial complementarity of effective information within the deep feature space. The highlight feature blind zone in a certain viewing angle can be automatically filled and corrected using effective non-blind zone features retained at the same physical coordinate position from other viewing angles. The global epidermal feature matrix data generated by the dimensionality reduction reconstruction is equivalent to a digital feature map that has removed optical reflection interference and possesses continuous texture information across the entire surface. This ensures that the data transmitted to the classification network is spatially continuous and complete, reduces the identification bias caused by the lack of local physical information, and improves the reliability of the system in identifying real defects.

[0330] Because the global epidermal feature matrix data extracted by the front-end convolutional neural network is an abstract high-dimensional mathematical quantity, physical actuators on industrial production lines (such as sorting valves) cannot directly read and understand this multi-dimensional tensor data. Furthermore, due to the complexity of the production environment, the model's predictions for certain edge-state jujubes (such as extremely minor scratches) may be ambiguous. If action commands are directly output based solely on the highest score, it can easily lead to malfunctions in the sorting hardware, reducing the yield of qualified products. Therefore, this paper proposes: performing a dimension-reduction inner product operation between the global epidermal feature matrix data and a preset classification mapping matrix data to obtain probability distribution vector data representing defect categories. When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output, as follows:

[0331] The global epidermal feature matrix data and the preset classification mapping matrix data are subjected to a dimension reduction inner product operation to obtain probability distribution vector data representing the defect category, specifically including:

[0332] The global epidermal feature matrix data is reconstructed into a one-dimensional vector and multiplied with a preset classification mapping matrix to obtain the original classification score data.

[0333] Perform natural exponentiation on each component of the original classification score data, and sum the results to obtain the exponent sum data;

[0334] Calculate the ratio of the natural exponentiation result of each component to the sum of the exponents, map the natural exponentiation result to the numerical range of zero to one, and output the probability distribution vector data.

[0335] Classification mapping matrix: A set of weight parameters pre-trained and fixed in a deep learning model (fully connected layer). Its function is to project the extracted spatial physical feature data onto a pre-defined specific category dimension (such as qualified fruit, bruised fruit, cracked fruit, etc.) through linear mapping.

[0336] Raw classification score data: The numerical value directly output after multiplying the feature vector and the classification mapping matrix (usually called logistic regression value or Logits). The domain of this value is from negative infinity to positive infinity, and it only reflects the relative magnitude of each category, without having absolute probabilistic statistical significance.

[0337] Natural exponentiation: The mathematical calculation process of raising a natural constant (approximately 2.718) to the power of a given value.

[0338] Probability distribution vector data: a one-dimensional array consisting of multiple components, each of which is strictly limited to the interval between zero and one, and the sum of all component values ​​equals one. Each component represents the confidence level that the target jujube belongs to the corresponding preset category.

[0339] Step 1: Matrix One-Dimensional Reconstruction and Original Score Calculation: The system reads the two-dimensional global epidermal feature matrix data into a one-dimensional vector structure according to a preset memory address order (e.g., row-major). Then, it performs matrix multiplication (inner product) with the preset classification mapping matrix. The reconstruction and inner product calculation formulas are as follows:

[0340] ;

[0341] ;

[0342] in, : Represents the first eigenvector in the reconstructed one-dimensional feature vector. Each component value. : Indicates the spatial coordinates of the two-dimensional global epidermal feature matrix data input for this step. The characteristic values ​​at the location. : Represents the index of a one-dimensional vector, with values ​​ranging from 1 to N. N: Represents the total number of elements contained in the global epidermal feature matrix (i.e., the product of spatial height and width). : indicates the calculated corresponding number The original classification score data for each category. : Represents the system's preset category index, with values ​​ranging from 1 to 1. ( (Total number of categories). : Represents the first element in the preset classification mapping matrix. Line number The column weights. Classification tasks are mathematically a form of dimensionality reduction mapping. Two-dimensional matrices preserve spatial location information, but classifiers require isotropic linear inputs. By flattening and inner products, information from the high-dimensional feature space can be projected into a low-dimensional space determined by the number of classes. This eliminates the spatial topology of the features, generating a continuous real score for each predefined class that integrates all local features.

[0343] Step 2: Perform natural exponentiation and summation: The system iterates through each component of the original classification score data, performs natural exponentiation on each component, and sums the exponentiation results for all categories. The formula for exponentiation and summation is:

[0344] ;

[0345] ;

[0346] in, : indicates the first The result of natural exponentiation for each category. : represents the natural constant, with a value of approximately 2.71828. : This represents the sum of exponents obtained by summing the results of exponent calculations for all categories. : Represents the index variable used to iterate through all categories, incrementing from 1 to the total number of categories. The raw scores contain negative numbers, and the differences may not be significant. The natural exponential function is a strictly monotonically increasing function that is always greater than zero. By converting all scores to positive numbers and utilizing the properties of the exponential function, the numerical differences between categories with higher and lower scores are amplified, thus enhancing the discriminative power of the classification.

[0347] Step 3: Calculate the ratio and output the probability distribution vector: The system iterates again, calculating the quotient of the exponent calculation result for each category and the sum of the overall exponents, completing the mapping of numerical intervals. The probability ratio mapping formula is:

[0348] ;

[0349] : Represents the probability distribution vector data corresponding to the first digit in the final output. The formula assigns probability component values ​​to each category. It transforms unbounded real numbers into a confidence level that conforms to statistical definitions. It rigorously maps the values ​​of all categories to the interval between 0 and 1, and guarantees... Based on this, the system outputs standardized probability distribution vector data, providing a unified comparison benchmark for subsequent threshold judgments that "meet the preset interception conditions".

[0350] Example 8: Continuing from the previous example, assume the global epidermal feature matrix data output by the previous step. For one The matrix. The coordinate values ​​are normalized and scaled to: (The real defect activation points retained above) (Background noise attenuated in the previous text) .

[0351] Assuming the system sets the total number of target categories Category 1 is "qualified fruit", and Category 2 is "target defective fruit".

[0352] Parameter value settings:

[0353] Preset classification mapping matrix data Contains a fixed weight with 4 rows and 2 columns:

[0354] Category 1 Weight Column: ;

[0355] Category 2 Weight Column: ;

[0356] (The weight settings here reflect the strong positive activation effect of the first feature point (1,1) on the "target defective fruit".)

[0357] Reconstructing a one-dimensional vector: Total number of elements .

[0358] Calculate the raw score for category 1:

[0359] .

[0360] Calculate the raw score for category 2:

[0361] .

[0362] Calculation results for each category index:

[0363] Category 1: .

[0364] Category 2: .

[0365] Calculate the sum of exponents:

[0366] .

[0367] Calculate the probability components of category 1:

[0368] .

[0369] Calculate the probability components of category 2:

[0370] .

[0371] Classification feature mapping and probability calculation results table:

[0372]

[0373] Finally, the system outputs probability distribution vector data [0.101, 0.899]. This data indicates that the confidence level of the currently detected jujube as a "target defective fruit" is as high as 0.899. This data will be passed to the hardware comparator, and once the interception condition is met (e.g., 0.85), the system will output sorting control signal data to execute the rejection.

[0374] When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output, specifically including:

[0375] Traverse the probability distribution vector data in memory address order, extract the component with the largest value as the highest confidence data, and extract the memory offset of the component as the target class address index.

[0376] Determine whether the target category address index is within the preset defect type address range;

[0377] If so, retrieve the preset interception threshold, calculate the difference between the highest confidence data and the preset interception threshold, and extract the sign bit of the difference data as the interception status identifier data;

[0378] When the interception status indicator data is a positive value, the preset sorting execution value is overwritten to the specified address, the data output status of the general input / output port is updated, and the updated data output status is used as the sorting control signal data output; when the interception status indicator data is a negative value, a preset silence command is triggered.

[0379] If not, trigger the branch jump instruction, skip the retrieval of the preset interception threshold, and simultaneously trigger the silent instruction.

[0380] Highest confidence data: The probability component with the largest value extracted from the probability distribution vector represents the probability value that the system is most certain about the current classification of winter jujubes.

[0381] Memory offset / target category address index: In contiguous memory space, the distance between the address of the highest confidence data and the starting address of the probability distribution vector. Since the probability values ​​of the categories are arranged in a fixed order, this memory distance (offset) is logically directly equivalent to the target category's digital identity index (ID).

[0382] Defect type address range: A pre-defined range of consecutive numerical indices in system memory. Addresses falling within this range correspond to categories that are considered defective and need to be eliminated (e.g., index 2 represents scratches, index 3 represents cracks).

[0383] The sign bit is the highest bit in the underlying binary data structure of a computer. After performing a subtraction operation, the value of this bit is used to represent the positive or negative state of the result (usually zero for positive numbers and one for negative numbers). This step uses it as an efficient Boolean criterion.

[0384] General Purpose Input / Output Port (GPIO): A hardware pin interface on the control motherboard used for electrical signal interaction with external physical actuators (such as sorting cylinders and high-pressure valves).

[0385] Silent command: A piece of low-level code sent to the control system to prevent the hardware state from changing. When this command is executed, general-purpose input / output ports maintain their default levels (such as low level), the sorting mechanism does not move, and the current jujubes pass through smoothly.

[0386] Step 1: Extracting the highest confidence score and address index based on memory traversal: The system reads each component of the probability distribution vector data generated in the previous step in ascending order of memory addresses, selects the component with the largest value through a comparator, and simultaneously records the memory offset of that component. The formula for extracting the maximum value and index is:

[0387] (in Traverse from 1 to )

[0388] ;

[0389] in, : Indicates the highest confidence level data extracted. : Represents the first probability distribution vector data. Each probability component. : Indicates the relative memory address index of the current traversal. : Indicates the total number of categories set by the system. : Represents the extracted target category address index (i.e., the one with the highest probability of generating the target category). (Value). The system must determine the predicted category and corresponding confidence level of the current jujube from a set of probability sequences across all possible states, thereby locking in the state with the highest probability of occurrence as the final decision benchmark.

[0390] Step 2: Range Determination Check: The system compares the obtained target category address index with the preset defect type address range in memory. The determination logic formula is as follows:

[0391] ;

[0392] in, : Indicates the Boolean result (true or false) of the address range determination. : Indicates the starting boundary index of the preset defect type address range. : Indicates the end boundary index of the preset defect type address range. : Represents a logical AND operation. The system category typically includes a "Normal Qualified Fruit" category (usually set to index 1). This normal category must first be logically separated from the other defective categories. The judgment check quickly filters out jujubes predicted to be normal, only allowing subsequent hardware triggering processes for jujubes initially judged to be defective.

[0393] Step 3: Threshold interception verification based on underlying subtraction operation:

[0394] When the above judgment result When true, the system retrieves the preset interception threshold, performs a subtraction operation using the underlying arithmetic unit, and reads the highest sign bit from the result register. The formula for extracting the difference and sign bit is:

[0395] ;

[0396] ;

[0397] in, : Represents the calculated difference data. : Indicates the pre-stored interception threshold. To ensure the rigor of sorting, this value is usually between 0.75 and 0.95. : Represents the interception status identifier data generated by extracting the sign bit of the difference data. In the underlying logic, if The sign bit represents the positive value; if The sign bit represents the negative value. This indicates the operation of retrieving the most significant bit of the underlying register. Industrial-grade microprocessors have long execution cycles for floating-point comparison instructions, and converting them into an operation of "subtraction followed by direct reading of the sign flag register" is the most efficient judgment method in computer architecture. Through a single operation, threshold blocking and status flag generation are completed simultaneously.

[0398] Step 4: Hardware status overwrite and control signal output:

[0399] Based on the acquired interception status identifier data, the system writes the corresponding machine code execution value to the general input / output port. The output logic and assignment formula are as follows:

[0400] when When the sign is positive: ;

[0401] when When the sign is negative: execute the silence command and maintain. ;

[0402] in, : Represents the data output status register of the general-purpose input / output port. : Indicates the preset sorting execution value (such as the machine hexadecimal code 0x01 representing a high level). : Indicates the preset silent state value (such as the machine hexadecimal code 0x00 representing a low level).

[0403] Step 5: Branch Jump Mechanism: When the interval determination result in step 2... When the condition is false (i.e., the target category is not within the defect range), the system triggers the underlying conditional jump instruction (Branch_Jump). The jump logic is as follows:

[0404] The program counter is forced to skip the threshold interception verification, hardware status overwriting, and control signal output operation addresses, and jump directly to the silent instruction address.

[0405] The majority of jujubes on the assembly line are qualified, so there is no need to perform subsequent defect interception threshold calculations for qualified fruit. By using short-circuit calculation paths, the controller's processor clock cycles are significantly reduced, ensuring that the system can meet the real-time requirements of high-throughput sorting lines.

[0406] Example 9: Continuing from the previous example, the system output probability distribution vector data in the previous stage. Assume the base memory address is 0x1000.

[0407] Category 1 (qualified fruit) is located at address 0x1000, with a relative offset index of 1.

[0408] Category 2 (target defective fruit) is located at address 0x1004, with a relative offset index of 2.

[0409] Parameter value settings:

[0410] The defect type address range is set to [2,2] (i.e.) , ).

[0411] Set preset interception threshold .

[0412] set up (High-level trigger for air-blowing rejection) (Low voltage silence release). Initial state .

[0413] Comparison values: 0.899 > 0.101.

[0414] Extracting the data with the highest confidence level: .

[0415] Extract the memory offset as an index: .

[0416] judge (i.e., 2) Whether it is located within the interval [2,2].

[0417] result If true (it is a defect category), no branch jump will be triggered.

[0418] Perform subtraction: .

[0419] Extract the sign bit: because Therefore, intercept status identifier data It is a positive value sign.

[0420] judge The value is positive, which meets the triggering condition.

[0421] Perform overwrite: Write to the specified address and update .

[0422] A high-level sorting control signal is output, the air valve opens, and the defective jujube is blown away.

[0423] Control group calculation (reflecting the effect of the interception condition): Assume another jujube is identified by the model, and its probability distribution is as follows: (At this point, the model's confidence in it being a defect wavers, and it is judged as a suspected defect).

[0424] Step 1 Extraction: , .

[0425] Second step judgment: If it is in the defect range [2,2], proceed to the third step.

[0426] Third step of subtraction: .

[0427] Extract the sign bit: because , sign bit It is a negative value sign.

[0428] Fourth step control: Trigger the silence command to maintain. The sorting mechanism remains inactive, safely releasing fruits that are suspected but not entirely certain, thus avoiding damage to good products.

[0429] State determination logic mapping table:

[0430]

[0431] Through the above process, the system finally completes a precise closed loop of conversion from software probabilistic analysis to physical electromechanical control.

[0432] In the aforementioned technology, abstract spatial features are compressed and projected into numerical values ​​corresponding to specific categories through feature flattening and the inner product operation of a preset mapping matrix. Subsequently, a normalization function based on the natural index is used to transform the unbounded classification scores into confidence vectors that strictly conform to a probability distribution. Finally, a joint interception condition mechanism is introduced at the hardware control end. This mechanism not only determines whether the highest probability falls within the defect category range but also mandates that the probability value must exceed a preset hard interception threshold, ultimately triggering a level reversal at the physical control port. The generation of the probability distribution vector provides a standardized confidence reference scale for each identification by the system; while the introduction of the interception threshold effectively filters out low-confidence noise generated when the model processes ambiguous samples, preventing the sorting mechanism from making erroneous rejections due to slight feature fluctuations. This dual confirmation mechanism improves the reliability of the final output sorting control signal data, ensuring the execution accuracy of the automated sorting production line.

[0433] A deep learning-based defect identification system for winter jujubes includes:

[0434] The data acquisition module is used to receive matrix data of multiple frames of images of the same target, winter jujube, collected at different spatial illumination angles in a time-division manner;

[0435] The specular mask generation module is used to calculate the brightness value and spatial gradient value of each coordinate element in the matrix data of each frame image, and extract the coordinate regions with brightness values ​​greater than a preset brightness threshold and spatial gradient values ​​less than a preset gradient threshold as specular blind zones, and generate corresponding specular suppression mask matrix data for the matrix data of each frame image.

[0436] The feature extraction and suppression module is used to extract features from the matrix data of each frame of the image based on the pre-stored convolution kernel weight matrix data, obtain the initial feature tensor data, and perform element-wise multiplication operation between the initial feature tensor data and the corresponding specular suppression mask matrix data to output the specular de-spectral feature tensor data.

[0437] The cross-dimensional feature fusion module is used to perform cross-dimensional splicing and recombination operations on the de-highlight feature tensor data output under different illumination angles, and fuse them to generate global epidermal feature matrix data representing the absence of reflective occlusion.

[0438] The defect classification and control output module is used to perform a dimension reduction inner product operation on the global skin feature matrix data and the preset classification mapping matrix data to obtain the probability distribution vector data representing the defect category. When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output.

[0439] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A method for identifying defects in jujubes based on deep learning, characterized in that, Includes the following steps: Receive matrix data of multiple frames of images of the same target, winter jujube, acquired at different spatial illumination angles in a time-division manner; Calculate the brightness value and spatial gradient value of each coordinate element in the matrix data of each frame image, and extract the coordinate regions with brightness values ​​greater than a preset brightness threshold and spatial gradient values ​​less than a preset gradient threshold as specular blind zones, and generate corresponding specular suppression mask matrix data for the matrix data of each frame image. Based on the pre-stored convolution kernel weight matrix data, feature extraction is performed on the matrix data of each frame of image to obtain initial feature tensor data. The initial feature tensor data is then multiplied element-wise with the corresponding specular suppression mask matrix data to output specular de-suppressed feature tensor data. The de-highlight feature tensor data output from different illumination angles are spliced ​​and recombined across dimensions to generate a global epidermal feature matrix data representing the absence of reflective occlusion. The global epidermal feature matrix data and the preset classification mapping matrix data are subjected to a dimension reduction inner product operation to obtain the probability distribution vector data representing the defect category. When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output.

2. The method for identifying defects in jujubes based on deep learning according to claim 1, characterized in that: Generate global epidermal feature matrix data representing the absence of reflective occlusion, specifically including: The high-dimensional cascaded tensor data is constructed by continuously arranging and splicing memory addresses along the channel depth dimension, including at least the de-highlight feature tensor data corresponding to the first illumination angle and the de-highlight feature tensor data corresponding to the second illumination angle. Obtain scalar weight vector data with the same number of two-dimensional channel feature matrices as the high-dimensional cascaded tensor data. Use the scalar weight vector data to perform multiplication and normalization operations on each two-dimensional channel feature matrix in the high-dimensional cascaded tensor data to obtain the weighted tensor data after compensation and allocation. Weighted summation and dimensionality reduction operations are performed on high-dimensional cascaded tensor data based on weighted tensor data to extract continuous edge features of areas not obscured by specular highlights, and output global epidermal feature matrix data.

3. The deep learning-based defect identification method for jujubes according to claim 2, characterized in that: The scalar weight vector data is dynamically generated by evaluating the feature distribution of the high-dimensional cascaded tensor data, specifically including: Traverse each two-dimensional channel feature matrix in the high-dimensional cascaded tensor data, and calculate the feature mean data of all coordinate elements in each two-dimensional channel feature matrix; The feature mean data of all two-dimensional channel feature matrices are arranged in the order of the physical arrangement of the channels and combined to generate a one-dimensional vector data of global illumination response that represents the current global illumination response state of winter jujube. Perform matrix multiplication between the one-dimensional vector data of the global illumination response and the preset first dimension reduction mapping matrix data to generate the initial inner product vector data; Iterate through each component value in the initial inner product vector data. When the component value is less than zero, overwrite the value in the corresponding memory address with zero. When the component value is greater than or equal to zero, keep the original value unchanged and output the transition vector data after non-linear truncation. Perform matrix multiplication between the transition vector data and the preset second-dimensional mapping matrix data to generate reconstructed vector data; For each component value in the reconstructed vector data, the power calculation is performed with the natural constant as the base and the opposite of the component value as the exponent. After adding the constant one to the calculation result, the reciprocal operation is performed to strictly converge all component values ​​to the value range of zero to one. Finally, a dynamic scalar weighted vector data with the same number of elements as the number of channels is output.

4. The deep learning-based defect identification method for jujubes according to claim 3, characterized in that: By performing product and normalization operations on the feature matrices of each two-dimensional channel in the high-dimensional cascaded tensor data using scalar weight vector data, the compensated weighted tensor data is obtained, specifically including: Analyze the dynamic scalar weight vector data and extract the scalar weight component values ​​corresponding to the feature matrix of each two-dimensional channel; For high-dimensional cascaded tensor data, we traverse each two-dimensional channel feature matrix within it, and perform point-by-point multiplication operations on the corresponding scalar weight component values ​​with the pixel feature values ​​at each coordinate position in the matrix, outputting primary weighted tensor data. Access the memory address where the dynamic scalar weight vector data is stored, sum all the component values ​​contained therein, and obtain the weight normalization factor data that represents the current global feature energy gain. Iterate through and calculate the ratio of each element value in the primary weighted tensor data to the weight normalization factor data, overwrite the calculation results to the preset memory buffer, and output the final compensated weighted tensor data.

5. The deep learning-based defect identification method for jujubes according to claim 4, characterized in that: Weighted summation and dimensionality reduction operations are performed on high-dimensional cascaded tensor data based on weighted tensor data to extract continuous edge features in areas not obscured by specular highlights, outputting global epidermal feature matrix data, specifically including: For the weighted tensor data after compensation and allocation, the spatial coordinates of the two-dimensional plane are used as the reference index for traversal, and multiple weighted feature component data corresponding to the same spatial coordinate position at all channel depths are extracted. The weighted feature component data extracted at the same spatial coordinate position are summed and the unique aggregated feature value corresponding to that spatial coordinate position is output. By traversing all spatial coordinate positions, the initial aggregated matrix data is obtained by compressing the dimension of the three-dimensional data structure into two dimensions. Get the preset edge response threshold, iterate through all elements in the initial aggregation matrix data, and determine whether each element is less than the edge response threshold. If so, it is determined to be smooth background or lighting shot noise, and the memory value of the corresponding element is multiplied by a preset attenuation coefficient for suppression. If not, it is determined to be a real defect edge activation point, and its memory value remains unchanged; The output is a global epidermal feature matrix data that enhances the response of effective edge activation points.

6. The method for identifying defects in jujubes based on deep learning according to claim 1, characterized in that: Generate corresponding specular suppression mask matrix data for the matrix data of each frame of the image, specifically including: Extract the pixel components of each coordinate element in the matrix data as brightness values; Retrieve the preset two-dimensional discrete difference operator matrix, perform window traversal with a sliding step of one on the matrix data, calculate the pixel difference between the current center coordinate element and its horizontal and vertical neighbor coordinate elements respectively, aggregate the absolute values ​​of the differences in each direction, and output the spatial gradient value corresponding to the current center coordinate element. When the brightness value of the coordinate element is greater than the preset brightness threshold and the spatial gradient value is less than the preset gradient threshold, the values ​​of the same relative coordinate addresses in the specular suppression mask matrix data are kept as the first weight constant used to block feature transmission. When the brightness value of a coordinate element is less than or equal to a preset brightness threshold, or the spatial gradient value is greater than or equal to a preset gradient threshold, the value at the corresponding address is overwritten with a second weight constant used to maintain feature propagation. Iterate through all coordinate elements in the matrix data and output the final specular suppression mask matrix data.

7. The deep learning-based defect identification method for jujubes according to claim 1, characterized in that: Output specular feature tensor data, specifically including: The addressing window, which is matched to the size of the convolution kernel weight matrix data, performs a two-dimensional sliding traversal on the image matrix data; The pixel data within each window is weighted and fused with the weight matrix, and the feature activation values ​​are output and cached sequentially to construct multi-channel three-dimensional initial feature tensor data. The two-dimensional specular suppression mask matrix data is mapped to the various channel levels of the three-dimensional initial feature tensor data to establish a co-positional coordinate mapping relationship. Traverse the memory physical addresses of the initial 3D feature tensor data, multiply the read feature activation values ​​with the corresponding coordinate mask weight constants, write the calculation results directly to the current memory address, and output the specular feature tensor data.

8. The method for identifying defects in jujubes based on deep learning according to claim 1, characterized in that: The global epidermal feature matrix data and the preset classification mapping matrix data are subjected to a dimension reduction inner product operation to obtain probability distribution vector data representing the defect category, specifically including: The global epidermal feature matrix data is reconstructed into a one-dimensional vector and multiplied with a preset classification mapping matrix to obtain the original classification score data. Perform natural exponentiation on each component of the original classification score data, and sum the results to obtain the exponent sum data; Calculate the ratio of the natural exponentiation result of each component to the sum of the exponents, map the natural exponentiation result to the numerical range of zero to one, and output the probability distribution vector data.

9. The method for identifying defects in jujubes based on deep learning according to claim 1, characterized in that: When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output, specifically including: Traverse the probability distribution vector data in memory address order, extract the component with the largest value as the highest confidence data, and extract the memory offset of the component as the target class address index. Determine whether the target category address index is within the preset defect type address range; If so, retrieve the preset interception threshold, calculate the difference between the highest confidence data and the preset interception threshold, and extract the sign bit of the difference data as the interception status identifier data; When the interception status indicator data is a positive value, the preset sorting execution value is overwritten to the specified address, the data output status of the general input / output port is updated, and the updated data output status is used as the sorting control signal data output; when the interception status indicator data is a negative value, a preset silence command is triggered. If not, trigger the branch jump instruction, skip the retrieval of the preset interception threshold, and simultaneously trigger the silence instruction.

10. A deep learning-based defect identification system for jujubes, characterized in that, include: The data acquisition module is used to receive matrix data of multiple frames of images of the same target, winter jujube, collected at different spatial illumination angles in a time-division manner; The specular mask generation module is used to calculate the brightness value and spatial gradient value of each coordinate element in the matrix data of each frame image, and extract the coordinate regions with brightness values ​​greater than a preset brightness threshold and spatial gradient values ​​less than a preset gradient threshold as specular blind zones, and generate corresponding specular suppression mask matrix data for the matrix data of each frame image. The feature extraction and suppression module is used to extract features from the matrix data of each frame of the image based on the pre-stored convolution kernel weight matrix data, obtain the initial feature tensor data, and perform element-wise multiplication operation between the initial feature tensor data and the corresponding specular suppression mask matrix data to output the specular de-spectral feature tensor data. The cross-dimensional feature fusion module is used to perform cross-dimensional splicing and recombination operations on the de-highlight feature tensor data output under different illumination angles, and fuse them to generate global epidermal feature matrix data representing the absence of reflective occlusion. The defect classification and control output module is used to perform a dimension reduction inner product operation on the global skin feature matrix data and the preset classification mapping matrix data to obtain the probability distribution vector data representing the defect category. When the probability data of the target defect category meets the preset interception conditions, the corresponding sorting control signal data is output.

Images