A live-streaming marketing robot for agricultural products based on origin traceability data mining

By registering the agricultural product live-streaming robot in its place of origin, compiling traceability fact units, planning display actions, and implementing multimodal synchronous control, the problem of inconsistency between the robot's broadcast content and the actual product at the place of origin was solved. Synchronous verification and continuous recovery were achieved, improving the accuracy and reliability of the live-streaming process.

CN123070967APending Publication Date: 2026-11-10GUANGXI WANJIN NEW ENERGY TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611229749.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-13
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing agricultural product live-streaming robots cannot autonomously determine their movement location, shooting objects, and broadcast content, which may not correspond to the objects displayed on-site. This results in inconsistencies between the broadcast content and the objects displayed on-site. Furthermore, they cannot simultaneously verify the current workstation, the current batch of agricultural products, and the current evidence carrier, which can easily lead to the continued broadcasting of erroneous batches and the breaking of the live-streaming evidence chain.

Method used

Through the origin operation space registration module, traceability fact unit compilation module, robot display action planning module, multimodal live broadcast synchronization control module, and broadcast physical consistency verification module, the correspondence between the agricultural product live broadcast robot and the on-site workstation, agricultural products and evidence carriers is realized. The robot's movement, framing, pointing and voice broadcast are controlled synchronously, and the live broadcast process is verified and restored in a timely manner.

Benefits of technology

This technology enables agricultural product live-streaming robots to move, film, and broadcast accurately at the place of origin, ensuring that the broadcast content is consistent with the actual products on site. It avoids the continued broadcasting of incorrect batches and the breakage of the live-streaming evidence chain, thereby improving the reliability and accuracy of the live-streaming process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123070967A_ABST
    Figure CN123070967A_ABST
Patent Text Reader

Abstract

This invention belongs to the field of agricultural product traceability data processing and intelligent robot control technology. It discloses an agricultural product live-streaming marketing robot based on origin traceability data mining, comprising an origin workspace registration module, a traceability fact unit compilation module, a robot display action planning module, a multimodal live-streaming synchronization control module, a broadcast physical consistency verification module, and an interaction interruption recovery and recording module. The invention binds origin traceability records with workstations, agricultural product batches, and evidence carriers to generate a traceability fact display task sequence, coordinating robot movement, framing, mechanical pointing, and voice broadcasting. Through on-site consistency verification and task breakpoint recovery, it maintains the consistency between the displayed object and the broadcast facts. This invention improves the verifiability, synchronization, and continuity of agricultural product origin live-streaming displays.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of agricultural product traceability data processing and intelligent robot control technology, specifically involving an agricultural product live-streaming marketing robot based on origin traceability data mining. Background Technology

[0002] As live-streaming marketing of agricultural products gradually extends from studio sales to planting bases, processing workshops, testing areas, and packaging areas, viewers not only focus on the price and appearance of agricultural products, but also need to verify the batch, production process, test results, and origin evidence. Therefore, live-streaming robots are required to move autonomously at the origin site and simultaneously display traceability records, workstations, physical agricultural products, and evidence carriers.

[0003] Existing agricultural product traceability technologies mainly rely on QR codes and centralized databases to store production, processing, testing, and logistics records. During live broadcasts, the host still manually queries the traceability records and explains them according to a pre-set script. The traceability records are not linked to the on-site workstations, the actual agricultural products, evidence carriers, and the robot's observation points, which makes it impossible for the live broadcast robot to autonomously determine its movement position, shooting objects, and broadcast content based on the traceability facts. This can easily lead to problems where the broadcast content does not correspond to the objects displayed on-site.

[0004] Existing live-streaming robots mainly perform path movement, camera tracking, mechanical pointing, and voice broadcasting, respectively. They lack collaborative control and on-site verification mechanisms for the origin traceability display process. When the robot switches workstations, is obscured by people, changes agricultural product batches, or the live-streaming interaction is interrupted, it cannot simultaneously verify the current workstation, the current agricultural product batch, and the current evidence carrier. It also cannot resume the original display process based on the task breakpoint, which can easily lead to the problem of incorrect batches being broadcast and the live-streaming evidence chain being broken. Summary of the Invention

[0005] This invention aims to provide a live-streaming marketing robot for agricultural products based on origin traceability data mining, which solves the problems of existing live-streaming robots broadcasting according to fixed scripts, the inability of traceability records to correspond with on-site workstations, agricultural products, and evidence carriers at the origin, and the asynchrony of robot movement, framing, pointing, and voice broadcasting.

[0006] To achieve the above objectives, this invention provides an agricultural product live-streaming marketing robot based on origin traceability data mining, including an origin operation space registration module, a traceability fact unit compilation module, a robot display action planning module, a multimodal live-streaming synchronization control module, a broadcast physical consistency verification module, and an interaction interruption recovery and recording module.

[0007] The origin operation space registration module registers the production stations, processing stations, testing stations, packaging stations, agricultural product batches, evidence carriers, and robot-observable locations in the origin operation site. It calculates the station space registration error based on the deviation between the robot positioning result and the station reference point, and generates an origin operation space registration data package.

[0008] The traceability fact unit compilation module receives the origin operation space registration data package and origin traceability record. It binds the agricultural product batch identifier, operation type, operation time, operation station, evidence summary and evidence carrier identifier in the origin traceability record with the corresponding on-site work station, agricultural product and evidence carrier. It calculates the executability of the fact based on the readable status of the evidence carrier, the reachability status of the operation station, the validity status of the traceability record and the consistency status of the agricultural product batch, and generates an executable traceability fact unit set.

[0009] The robot demonstration motion planning module receives the origin workspace registration data package and the set of executable traceability fact units. It calculates the passage cost between workstations based on the path length between workstations, the distance between obstacles, the number of robot turns, and the demonstration preparation time. It determines the demonstration order according to the operation time sequence and process connection relationship between the executable traceability fact units. It configures movement actions, framing actions, evidence reading actions, mechanical pointing actions, and voice broadcasting actions for each executable traceability fact unit, generating a traceability fact demonstration task sequence.

[0010] The multimodal live streaming synchronization control module receives the task sequence for tracing facts and displays them, and controls the mobile base, lifting pan-tilt unit, main camera, local camera, fill light, near-field tag reader, mechanical pointing arm, and speaker to perform corresponding display actions. It calculates the visual quality of the image based on the proportion of the actual agricultural product in the image, the clarity of the evidence carrier, the degree of target occlusion, and the degree of motion blur. It calculates the multimodal synchronization deviation based on the image stabilization time, the mechanical pointing time, and the start time of the voice broadcast, and generates a multimodal display execution data package.

[0011] The broadcast physical consistency verification module receives the multimodal display execution data packet and the set of executable traceability fact units, extracts the physical characteristics of agricultural products in the live broadcast, rereads the on-site evidence carrier, and obtains the robot's current workstation. It compares the actual obtained agricultural product batch identifier, evidence carrier identifier, current workstation, and display time with the current executable traceability fact unit, calculates the on-site consistency deviation, generates consistency verification results and display correction instructions, and feeds back the display correction instructions to the robot display motion planning module and the multimodal live broadcast synchronization control module.

[0012] The Interaction Interruption Recovery and Recording Module receives live interactive questions, the sequence of source tracing facts display tasks, consistency verification results, and multimodal display execution data packets. It calculates the user's question interruption handling priority based on the correlation between the live interactive questions and the currently executable source tracing facts units, the number of times the questions are repeated, the timeliness of the questions, and the current action security status. Before executing the question response task, it saves the task breakpoint. After the question response is completed, it re-executes the workstation positioning, evidence carrier reading, and physical consistency verification of the broadcast. Based on the verification results, it restores the source tracing facts display task sequence and generates a live marketing execution record.

[0013] Furthermore, the consistency verification results include a consistent state, a correctable deviation state, and a prohibition on continuing broadcasting state. When the on-site consistency deviation does not reach the first correction threshold, it is determined to be in a consistent state. When the on-site consistency deviation reaches the first correction threshold but does not reach the second suspension threshold, it is determined to be in a correctable deviation state. When the on-site consistency deviation reaches the second suspension threshold or the agricultural product batch identification is inconsistent, it is determined to be in a prohibition on continuing broadcasting state.

[0014] This invention establishes a robot-executable display chain around the actual agricultural products, workstations, evidence carriers, and traceability records at the place of origin, so that the facts of the place of origin broadcast during the live broadcast can be jointly verified by the current workstation, the current actual product, and the current evidence carrier.

[0015] The beneficial effects of this invention are as follows: (1) By utilizing the principle of registration of origin operation space and compilation of traceability fact units, the batch identifier of agricultural products, operation time, operation position and evidence carrier identifier in the origin traceability record are bound to the on-site operation position, agricultural products, evidence carrier position and robot observation position, realizing the transformation of traceability record into robot executable display task, and solving the problem that fixed live script cannot accurately correspond to the on-site physical objects and evidence carriers of the origin.

[0016] (2) By utilizing the principles of inter-station access cost, visual quality, and multimodal synchronous deviation collaborative control, the movement action, framing action, evidence reading action, mechanical pointing action, and voice broadcasting action are uniformly arranged. The execution order is corrected according to the screen stabilization time, mechanical pointing arrival time, and voice broadcasting start time, thus realizing the synchronous execution of robot movement, on-site framing, object pointing, and fact broadcasting. This solves the problem of misalignment between screen, pointing object, and broadcasting content caused by the independent operation of different live broadcast execution mechanisms.

[0017] (3) By utilizing the principle of consistency verification of broadcast physical objects and task breakpoint recovery, the batch identification of agricultural products, evidence carrier identification, current workstation and display time obtained at the live broadcast site are compared with the current executable traceability fact unit. After the live broadcast interaction ends, the workstation positioning, evidence carrier reading and consistency verification are re-executed, which realizes the timely termination of erroneous display and the continuous recovery of interrupted tasks. This solves the problems of erroneous traceability facts being broadcast after agricultural product batch change, workstation switching and interaction interruption and the breakage of the live broadcast evidence chain. Attached Figure Description

[0018] The accompanying drawings are provided to further understand the present invention and form part of the specification. They are used together with the embodiments of the present invention to explain the invention and do not constitute a limitation thereof.

[0019] Figure 1 This is a schematic diagram of the module data chain of the agricultural product live-streaming marketing robot of the present invention; Figure 2 The flowchart for the execution and consistency verification of this invention is shown. Detailed Implementation

[0020] Example 1, see Figures 1-2 The present invention provides an agricultural product live-streaming marketing robot based on origin traceability data mining, including an origin operation space registration module, a traceability fact unit compilation module, a robot display action planning module, a multimodal live-streaming synchronization control module, a broadcast physical consistency verification module, and an interaction interruption recovery and recording module.

[0021] This embodiment is applied to a live-streaming scenario of a Yunnan small-bean coffee processing base. The processing base is set up along the robot's travel route with a fresh fruit receiving station, a flotation station, a peeling station, a fermentation station, a sun-drying station, a moisture content detection station, a grading station, and a finished product storage station. The live-streaming marketing robot is equipped with a mobile chassis, LiDAR, a lifting gimbal, a main camera, a close-up camera, a microphone, a speaker, a supplementary lighting component, a near-field tag reader, a mechanical pointer arm, and an edge computing terminal.

[0022] The mobile chassis moves within the processing base corridor, the lidar acquires the corridor boundaries and workstation positions, the lifting gimbal adjusts the shooting height of the main camera and the close-up camera, the mechanical pointer arm points to the coffee cherries, fermentation tank, sun-drying bed, moisture content meter and finished product packaging label being explained, and the edge computing terminal receives origin traceability data and controls the movement, shooting, broadcasting, pointing and questioning actions.

[0023] The origin operation space registration module receives base floor maps, workstation identifiers, equipment locations, robot positioning data, and agricultural product traceability data. It maps planting plots, fresh fruit receiving batches, processing workstations, testing equipment, and storage locations to the same robot navigation coordinate system, creating an origin space task map that includes spatial nodes, access edges, and display conditions.

[0024] The traceability fact unit compilation module receives the origin spatial task map, extracts fact fields from agricultural product traceability data that can be verified through on-site physical objects, equipment readings and label information, and combines fact fields, data sources, associated workstations, broadcast text slots, display objects and allowed usage time into traceability fact units to generate a traceability fact unit set.

[0025] The robot demonstration motion planning module receives the set of traceability fact units. Based on the workstation location, display object size, channel occupancy status, robot remaining battery power, and live broadcast duration associated with each traceability fact unit, it determines the workstation access order, chassis movement path, gimbal shooting posture, mechanical pointer arm movement, and broadcast time period, and generates a robot demonstration motion sequence.

[0026] The multimodal live streaming synchronization control module receives the robot's action sequence and controls the chassis docking, gimbal turning, camera switching, mechanical indicator arm pointing, sensor reading overlay, and voice broadcasting according to a unified live streaming timeline. This ensures that the broadcast content appears in the same live streaming segment as the corresponding displayed object and on-site readings, generating a multimodal live streaming synchronization frame.

[0027] The physical consistency verification module receives multimodal live synchronous frames and reads the current workstation label, packaging traceability code, equipment display value and visual recognition result before and after the voice broadcast. It compares the content read on site with the agricultural product batch, fact field and numerical range in the traceability fact unit to generate the physical consistency verification result.

[0028] The interactive interruption recovery and recording module receives the results of the physical consistency verification and questions from live users. It inserts questions that can be answered by the current traceability fact unit set into the robot's action sequence. After the answer is completed, it determines the recovery position based on the interrupted action, the current workstation status, and the completed broadcast content. It then combines the actual action, broadcast text, on-site verification content, and interruption recovery process into a robot live execution record.

[0029] When the physical consistency verification result is qualified, the interaction interruption recovery and recording module allows the robot to continue broadcasting the corresponding traceability fact unit. When the on-site agricultural product batch, equipment reading, and packaging label are inconsistent with the traceability fact unit, the robot stops broadcasting the corresponding fact, keeps the camera pointed at the current verification object, and generates an on-site review prompt.

[0030] The system outputs live robot footage, source tracing information, on-site verification footage, user question answers, and live robot execution records. It does not generate recommended agricultural product batch sequences, nor does it change agricultural product batches based on user transaction probability. The live stream audience is determined by the live stream tasks pre-set by the processing base.

[0031] Through the above processing, this system forms a continuous data processing chain of "origin spatial task map - traceability fact unit set - robot display action sequence - multimodal live broadcast synchronization frame - broadcast physical consistency verification result - robot live broadcast execution record", which enables origin traceability data to be converted into robot-executable movement, shooting, instruction and broadcasting actions, solving the problem that traditional live broadcast robots can only broadcast according to a fixed script and cannot synchronously display on-site workstations, physical batches and traceability facts.

[0032] Example 2: Based on all the above examples, the origin operation space registration module specifically includes a base map construction unit, a workstation identifier parsing unit, a traceability object binding unit, a coordinate transformation calibration unit, a passage edge generation unit, and a spatial task map updating unit.

[0033] The base map construction unit uses LiDAR to acquire information about the processing base's passageways, walls, equipment outlines, temporary material storage areas, and personnel work areas. It then generates a two-dimensional grid map in the robot navigation coordinate system and establishes spatial nodes for fresh fruit receiving stations, flotation stations, peeling stations, fermentation stations, sun-drying stations, moisture content detection stations, grading stations, and finished product storage stations.

[0034] The workstation identification and resolution unit reads the visual positioning marks and near-field electronic tags set at each workstation. The visual positioning marks provide the direction and distance of the workstation relative to the camera, and the near-field electronic tags provide the workstation number, equipment number, and process step number. When the numbers of the two tags match, the corresponding position is determined as the workstation reference position.

[0035] The traceability object binding unit binds the planting plot number, fresh fruit receiving record, fermentation tank number, sun-drying bed number, moisture content test record, grading record, and packaging traceability code to the corresponding spatial nodes according to the coffee batch identifier, so that each traceability data has a place for on-site display.

[0036] The coordinate transformation calibration unit selects no fewer than three visual positioning marker feature points at each workstation, transforms the local coordinates of the workstation to the robot navigation coordinate system, and calculates the spatial registration error of the i-th workstation according to Formula 1:

[0037] In Formula 1, This represents the spatial registration error of the i-th workstation. This represents the number of visual positioning marker feature points participating in the calibration at the i-th workstation, where m represents the feature point number. This represents the recognition sharpness weight corresponding to the m-th feature point. This represents the coordinates of the m-th feature point in the robot navigation coordinate system. This represents the coordinates of the m-th feature point in the local coordinate system of the workstation. This represents the coordinate rotation matrix of the i-th workstation. This represents the coordinate translation vector of the i-th workstation. This represents the Euclidean distance.

[0038] If the spatial registration error is no more than 50mm, the workstation spatial node is saved. If the spatial registration error is greater than 50mm, the visual positioning mark is reread. If the requirement is still not met after two consecutive rereads, the corresponding workstation is set to a non-automatic docking state, and the robot can only take distant pictures from a safe position outside the workstation.

[0039] The passage edge generation unit connects adjacent workstation space nodes and records the passage length, minimum passage width, turning angle, personnel crossing frequency, ground slope, and wireless network signal strength along each passage edge. No passage edge is established when the passage width is less than the robot's safe passage width.

[0040] The space mission map update unit calculates the travel cost between the i-th workstation and the j-th workstation according to Formula 2:

[0041] In Formula 2, Let $\frac{i}{j}$ represent the travel cost between workstation $i$ and workstation $j$. This indicates the travel distance between two workstations. This indicates the allowed robot movement speed for the corresponding passable edge. This indicates the heading angle that the robot needs to adjust when moving to the next workstation. This indicates the permissible steering angular velocity of the mobile chassis. This indicates the degree of overlap of people on the passageway during the current time period. This indicates the availability of the wireless network on the passing edge. This indicates the influence of the ground slope on the passageway. The correction factor indicating the heading adjustment time. A correction factor representing the risk of cross-infection among personnel. A correction factor representing the risk of wireless network outages. This represents the correction factor for the effect of ground slope.

[0042] When staff, transfer vehicles, and temporary materials occupy the passageway, the spatial task map update unit increases the degree of personnel crossover and passage cost on the corresponding passage side, and the robot's motion planning module reselects the workstation access route based on the updated origin spatial task map.

[0043] The origin spatial task map includes spatial nodes, workstation reference locations, associated agricultural product batches, display objects, photographable areas, access edges, access costs, and non-automatic docking status. The traceability fact unit compilation module reads traceability data that can be displayed on-site at the corresponding workstation based on the spatial nodes.

[0044] Through the above processing, the origin operation space registration module not only records the name of the processing base and the workstation number, but also binds the agricultural product traceability data to the actual workstation that the robot can reach and photograph. After receiving a traceability fact, the robot can determine the location of the corresponding physical object and the safe passage route.

[0045] Example 3: Based on all the above examples, the fact unit compilation module specifically includes a fact field parsing unit, a field display condition matching unit, a fact timeliness control unit, a fact text compilation unit, a disabled expression filtering unit, and a fact unit encapsulation unit.

[0046] The traceability field parsing unit reads the traceability data of the coffee batch corresponding to the current live broadcast task and extracts the plot altitude, coffee variety, harvest date, fresh fruit weight, fermentation start time, fermentation duration, sun-drying bed number, current moisture content, grading results, packaging date, and storage location.

[0047] The on-site display condition matching unit configures display objects that can be verified on-site for each traceability field based on the origin spatial task map. The fresh fruit weight corresponds to the electronic scale at the fresh fruit receiving station, the fermentation duration corresponds to the fermentation tank control terminal, the current moisture content corresponds to the moisture content detector, the grading result corresponds to the grading station screen and grading label, and the packaging date corresponds to the finished product packaging traceability code.

[0048] The fact timeliness control unit sets different allowable usage times for static facts, process facts, and real-time facts. Plot altitude and coffee variety are set as static facts, harvest date and fermentation duration are set as process facts, and fresh fruit weight and current moisture content are set as real-time facts. Real-time facts are not used after the last collection time of the corresponding device.

[0049] The on-site demonstration showed that the condition matching unit calculated the factual executability of the u-th candidate tracing field according to Formula 3:

[0050] In Formula 3, This represents the factual executability of the u-th candidate tracing field. Indicates the completeness of the data in the candidate tracing field. This indicates the consistency status between the candidate traceability fields and the agricultural product batches in the current live streaming task. This indicates the time interval between the candidate traceability field and the most recent collection time. This indicates the allowed usage time for the corresponding traceability field. This indicates a disabled state when there are conflicts in the candidate tracing fields. This indicates that the candidate traceability field has a photographable physical state. This indicates the status that the candidate traceability fields can be read by field devices. This indicates the status of the candidate tracing field being readable through the packaging label. The contribution coefficient representing the physical display status. The contribution coefficient representing the device's read status. This represents the contribution coefficient of the tag reading status.

[0051] When the fact executability reaches the fact compilation threshold, the fact text compilation unit converts the candidate traceability fields into broadcast text slots. The broadcast text slots retain the agricultural product batch, data value, unit of measurement, collection time, and on-site display object, and do not directly write the data value into the unmodifiable fixed voice script.

[0052] Disable expression filtering units to create a set of absolute marketing expressions and a set of efficacy expressions lacking data sources. When a corresponding expression appears in the generated text, delete the corresponding sentence or paragraph. Do not use "highest quality," "absolutely natural," "cure diseases," and "permanently authentic" as content broadcast by the robot.

[0053] The fact text compilation unit generates the broadcast structure in the order of "on-site object pointing - fact field reading - value and unit broadcasting - data time description - on-site verification prompt". Taking the current moisture content as an example, the robot first points the camera at the moisture content detector, then reads the displayed value, and then broadcasts the current displayed value and the detection time, without presenting the historical detection value as the current detection value.

[0054] The fact unit encapsulation unit encapsulates the agricultural product batch identifier, fact field, data value, unit of measurement, data source, allowed usage time, associated workstation, display object, broadcast text slot, verification method, and disabled status into traceability fact units, and generates a traceability fact unit set according to the current live broadcast task.

[0055] Regarding parameter adjustments: shorten the allowable usage time for real-time facts with shorter processing cycles; increase the contribution coefficient of device reading status when the device network is stable; increase the contribution coefficient of label reading status when packaging labels are the main source of verification; and decrease the contribution coefficient of physical display status when visual obstruction is frequent.

[0056] Through the above processing, the source fact unit compilation module converts the database fields into data objects that simultaneously contain broadcast content, on-site location, verification method, and valid time. This prevents the live broadcast robot from reading values ​​from the database and broadcasting them directly without the on-site object, and also prevents expired data and data lacking display conditions from entering the robot's display action sequence.

[0057] Example 4: Based on all the above examples, the robot demonstration motion planning module specifically includes a live streaming task segmentation unit, a workstation access sorting unit, a docking position generation unit, a shooting pose calculation unit, a mechanical instruction motion generation unit, and a motion conflict resolution unit.

[0058] The live broadcast task is divided into segments based on the introduction of the production environment, the display of fresh fruit receiving, the display of processing technology, the display of moisture content testing, the display of grading, and the display of packaging traceability. Each live broadcast task segment is associated with at least one traceability fact unit and one spatial node.

[0059] The station access sequencing unit determines the order in which the robot visits the fresh fruit receiving station, fermentation station, sun-drying station, moisture content detection station, grading station, and finished product storage station based on the access cost in the origin spatial task map and the allowed execution time of each live-streaming task segment.

[0060] When the fermentation station is performing material feeding and cleaning operations, the station access sorting unit postpones the corresponding live broadcast task segment, first guiding the robot to the moisture content detection station and the finished product storage station, and then returning to the fermentation station after the fermentation station is restored to a state where it can be filmed.

[0061] The docking position generation unit generates multiple candidate docking positions within the camera-capable area of ​​each workstation. The shooting pose calculation unit calculates the visual quality of each candidate docking position based on the size of the displayed object, the occlusion contour, the lens focal length, the gimbal angle, and the robot's safe distance.

[0062] The shooting pose calculation unit calculates the image visual quality at the t-th shooting moment according to Formula 4:

[0063] In Formula 4, This represents the visual quality of the image at the t-th shooting moment. This indicates the proportion of the projected area of ​​the displayed object to the live broadcast screen. This indicates the normalized sharpness of the displayed object's edges. This indicates the proportion of the displayed object that is obscured by people, equipment parts, and mechanical indicator arms. This represents the angle between the camera's optical axis and the normal to the front of the object being displayed. This indicates the proportion by which the displayed object extends beyond the safe boundaries of the image. This indicates the impact of the shooting angle on the readability of the front view.

[0064] The camera pose calculation unit selects the candidate docking position with the highest visual quality and meets the robot's safety distance requirements. The packaging traceability code and the detector display screen are captured by a close-up camera, while the workstation panorama and processing actions are captured by the main camera.

[0065] The mechanical instruction motion generation unit generates lifting, rotating, extending, pausing, and retracting actions of the mechanical instruction arm based on the position of the displayed object in the robot coordinate system. The end of the mechanical instruction arm maintains a distance of more than 150mm from the displayed object and does not come into contact with agricultural products, testing equipment, or rotating parts.

[0066] The motion conflict resolution unit checks the motion timing of the mobile chassis, lifting pan-tilt unit, camera, and mechanical pointer arm. When the mobile chassis is in motion, the mechanical pointer arm remains in a retracted state. When the mechanical pointer arm approaches the camera's field of view, the instruction action is delayed. When a worker enters the robot's safe area, the chassis and mechanical pointer arm are paused.

[0067] The robot's demonstration action sequence includes the live task segment, target workstation, chassis path, docking position, gimbal attitude, camera type, mechanical instruction action, broadcast text slot, action start time, action end condition, and abnormal exit position.

[0068] Through the above processing, the robot's motion planning module generates executable actions based on the actual passages, workstation occupancy status, and display object location of the processing base. It does not move between workstations in a fixed order, enabling the robot to complete live streaming of the origin without affecting coffee processing production.

[0069] Example 5: Based on all the above examples, the multimodal live streaming synchronization control module specifically includes a unified live streaming clock unit, an action triggering unit, a voice broadcast control unit, a video display control unit, a live reading overlay unit, a network latency compensation unit, and a synchronization anomaly handling unit.

[0070] The unified live broadcast clock unit assigns unified timestamps to the mobile chassis, lifting pan-tilt unit, main camera, close-up camera, mechanical indicator arm, voice broadcaster, and on-site reading overlay process, and converts the start time of the action sequence in the robot's demonstration into the trigger time on the unified live broadcast timeline.

[0071] The motion triggering unit initiates gimbal turning after the robot reaches the target parking position and the chassis speed drops to zero. After the gimbal attitude reaches the target range, the camera switching is initiated. After the camera image reaches the required visual quality, mechanical instruction actions and voice broadcasts are initiated.

[0072] The voice broadcast control unit writes the latest data value and acquisition time read on-site into the broadcast text slot. After the writing process is completed, it generates the statement to be broadcast. When the on-site reading fails, it does not use the old value in the database to replace the current value, but instead changes the corresponding statement to an on-site reading failure prompt.

[0073] The on-site reading overlay unit converts the readings from the moisture content meter, electronic scale, and fermenter control terminal into overlay fields on the live screen, and records the collection time and associated traceability facts for the overlay fields.

[0074] The network latency compensation unit calculates the multimodal synchronization deviation of the t-th live segment according to Formula 5:

[0075] In Formula 5, This represents the multimodal synchronization deviation of the t-th live segment. This indicates the timestamp of the fact field appearing in the voice broadcast. Indicates the timestamp of the corresponding displayed object entering the valid screen. This indicates the timestamp at which the on-site reading was written to the live stream screen. This indicates the timestamp indicating that the robotic pointer arm has completed its pointing motion. This indicates the amount of data to be sent when a live stream segment enters the sending queue. Indicates the current effective upload speed of the network. This indicates the additional waiting time caused by the send queue.

[0076] When the multimodal synchronization deviation is no more than 500ms, a multimodal live streaming synchronization frame is generated. When the multimodal synchronization deviation is greater than 500ms but no more than 1500ms, the voice broadcast is delayed while the current screen is maintained. When the multimodal synchronization deviation is greater than 1500ms, the broadcast of fact fields is stopped and the workstation description segment without real-time values ​​is switched.

[0077] When the effective network upload rate decreases, the video display control unit prioritizes maintaining the clarity of the packaging traceability code and the equipment display value in the local close-up camera, reduces the frame rate of the workstation panoramic view, and does not reduce the readability of the characters in the on-site verification screen.

[0078] The synchronous anomaly handling unit pauses the broadcast of fact fields when a person obscures the displayed object, recalculates the visual quality of the image after the image is restored, keeps the camera recording and stops mechanical instruction actions when the chassis positioning drifts, and saves local video and the execution progress of the robot's display action sequence when the network is continuously interrupted.

[0079] Regarding parameter adjustments: shorten the sensor reading cycle when the device display value changes rapidly; increase the sending queue reserve when network jitter increases; increase the resolution of the local close-up camera when the packaging traceability code size decreases; and extend the image stabilization confirmation time when personnel flow increases.

[0080] Through the above processing, the multimodal live streaming synchronization control module ensures that the "facts spoken by the robot, the objects captured, the locations pointed to, and the superimposed readings" are all in the same live streaming segment. This prevents the robot from broadcasting data from the previous workstation after it has switched to the next workstation, and also prevents network latency from causing mismatches between on-site readings and on-screen objects.

[0081] Example 6: Based on all the above examples, the broadcast physical consistency verification module specifically includes a current batch locking unit, a label character recognition unit, a device reading extraction unit, a visual attribute extraction unit, a factual difference calculation unit, and a verification action execution unit. The interactive interruption recovery and recording module specifically includes a question semantic parsing unit, an answer resource retrieval unit, an interruption priority calculation unit, an action safety convergence unit, a recovery anchor point generation unit, and an execution record encapsulation unit.

[0082] The current batch locking unit reads the near-field electronic tag and packaging traceability code of the current workstation to obtain the on-site agricultural product batch identifier, and compares it with the agricultural product batch identifier in the traceability fact unit that is being executed. If the two identifiers are inconsistent, writing to the broadcast text slot is prohibited.

[0083] The label character recognition unit extracts coffee variety, processing method, packaging date and batch number from the packaging label; the equipment reading extraction unit extracts values ​​and units of measurement from the electronic scale, fermentation tank control terminal and moisture content meter; and the visual attribute extraction unit extracts the color ratio of fresh fruit, sun-drying bed number, packaging appearance and grading label from the main camera and close-up camera images.

[0084] The factual discrepancy calculation unit calculates the field consistency deviation of the u-th traceability fact unit according to Formula 6:

[0085] In Formula Six, This represents the field consistency deviation of the u-th traceability fact unit. This indicates the batch identifier of agricultural products recorded in the traceability fact unit. This indicates the batch label of the agricultural product read on-site. This indicates the quantity of identical batches of agricultural products, where 1 represents the batch number and 0 represents the batch number that does not match. This represents the numerical value recorded by the fact unit for tracing the source. This represents the value read by the field device. This indicates the allowed range of numerical variations for the corresponding fact field. This represents a stable term to prevent the denominator from being zero. This indicates the degree of character matching between the characters on the packaging label and the traceability fact field. This indicates the degree of image matching between on-site visual attributes and source fact fields. The contribution coefficient representing the batch identification difference. The contribution coefficient representing the numerical difference. The contribution coefficient representing the difference in label characters. The contribution coefficient representing the difference in visual attributes.

[0086] When the on-site consistency deviation is not greater than the consistency threshold, a verification qualified status is generated. When the on-site consistency deviation is greater than the consistency threshold and the batch identifier is consistent, the device value and label character are reread. When the on-site consistency deviation is greater than the consistency threshold and the batch identifier is inconsistent, the fact broadcast is stopped immediately and a batch switching prompt is generated.

[0087] If the verification action execution unit still has numerical differences after rereading, it keeps the camera at the position of the device display screen and announces that there is a difference between the displayed value and the recorded value. It does not make an automatic conclusion on the cause of the difference, and writes the difference data into the robot live execution record.

[0088] The question semantic parsing unit extracts the question object, fact fields, time reference, and relevance to the current workstation from the user's question. The answer resource retrieval unit retrieves the answer content only from the traceability fact unit set and the broadcast physical consistency verification results, and does not generate factual answers from the network text that has not been bound to the current agricultural product batch.

[0089] The interrupt priority calculation unit calculates the interrupt handling priority of the q-th user question according to Formula 7:

[0090] In Formula 7, This indicates the interruption handling priority for the q-th user's problem. A_q represents the degree of relevance between the user's question and the current source fact unit, and A_q represents the extent to which the source fact unit set can provide a complete answer. This indicates the extent to which the current scene footage can support the content of the answer. This indicates the waiting time after a user's issue enters the issue queue. Indicates the effective response time for user issues. This represents the cost of interrupting the current robot action. This indicates the safety implications of interrupting the current robot action. The contribution coefficient, representing the degree of relevance to the problem, The contribution coefficient representing the completeness of the response. The contribution coefficient, representing the degree of support from the on-site footage. The coefficient representing the contribution of the problem's waiting time. This represents the deduction factor for movement costs. This represents the deduction factor for safety impact.

[0091] When the interruption handling priority reaches the answer threshold and the robot is in a stationary shooting state, the motion safety convergence unit pauses the current voice broadcast and maintains the camera position, the answer resource retrieval unit generates the answer content, and after the answer is completed, the recovery anchor point generation unit determines the recovery position based on the completed broadcast text slot and the on-site workstation status.

[0092] When the robot is traversing a passageway, the mechanical pointer arm is extending, or a worker enters a safe area, even if the interruption priority reaches the response threshold, the question will not be answered immediately. The question will remain in the question queue, and the robot will only be able to answer after it has safely retracted and entered a stationary state.

[0093] The recovery anchor points include the interrupted traceability fact unit, the sequence number of the completed statement, the currently displayed object, the camera type, the status of the mechanical indicator arm, and the remaining time of the workstation stay. During recovery, the completed values ​​are not replayed, and the on-site verification actions that have not yet been executed are not skipped.

[0094] The execution record encapsulation unit encapsulates the robot's movement path, workstation docking position, traceability fact unit, actual broadcast text, on-site labels, equipment readings, multimodal synchronization deviation, on-site consistency deviation, user questions, answer content, and recovery anchor points into a robot live broadcast execution record.

[0095] Through the above processing, the product consistency verification module compares the database fields with the on-site batch, equipment display and packaging label in real time before the fact broadcast. The interaction interruption recovery and recording module then handles user questions according to the robot's safety status, so that the live broadcast robot can answer questions related to the current place of origin of the product, and will not cause the movement, shooting and fact verification process to be disordered due to the insertion of questions and answers.

[0096] Example 7: This example is based on all the above examples and uses a live broadcast of the coffee harvest season at a Yunnan small-bean coffee processing base to illustrate the system's operation.

[0097] The processing base is divided into a fresh fruit receiving area, a fermentation area, a sun-drying area, a testing area, a grading area, and a finished product storage area. The robot's route is 126m long, the minimum width of the passage is 1.8m, the width of the mobile chassis is 0.72m, and the coffee batch number corresponding to the live broadcast task is CF-YN-0826.

[0098] The origin operation space registration module uses LiDAR to create a two-dimensional grid map of the processing base and reads the visual positioning marks and near-field electronic tags of each workstation, binding the electronic scale, No. 2 fermentation tank, No. 7 sun-drying bed, moisture content detector, grading label and finished product location to the corresponding space node.

[0099] When the system performs spatial registration calibration, the spatial registration error of the moisture content detection station is 32mm, which meets the automatic docking requirements. One side of the No. 7 sun-drying bed is blocked by a temporary turnover basket. The spatial task map update unit increases the passage cost of the corresponding passage side. The robot demonstration motion planning module adjusts the sun-drying station to the later part of the live broadcast task.

[0100] The traceability fact unit compilation module extracts the altitude of 1650m, harvest date, fresh fruit receiving weight, No. 2 fermentation tank number, fermentation duration, No. 7 sun-drying bed number, current moisture content, grading results, and packaging date from the traceability data of coffee batch CF-YN-0826, and binds them to the planting plot display board, electronic scale, fermentation tank control terminal, sun-drying bed identification, moisture content meter, and finished product packaging traceability code, respectively.

[0101] The current moisture content is a real-time fact. The system reads that the moisture content detector's most recent data acquisition time is 3 minutes from the current live broadcast time. The data is within the allowable usage time, and the moisture content detector can enter the field of view of the local close-up camera. The traceability fact unit compilation module compiles the corresponding fields into an executable traceability fact unit.

[0102] The robot moves from its starting position to the fresh fruit receiving station. After reaching the docking position, it stops moving its chassis, raises the pan-tilt head, and points the main camera at the fresh fruit receiving area. The mechanical indicator arm points to the electronic scale, and the voice broadcast control unit writes the on-site display value of the electronic scale into the broadcast text slot and then completes the broadcast.

[0103] Before entering the fermentation zone, the robot detected that staff were feeding materials into fermentation tank No. 1. The passageway of fermentation tank No. 2 in the origin space mission map was still passable. The robot reduced its movement speed and entered the parking position of fermentation tank No. 2 through the safety passage, without taking pictures of fermentation tank No. 1 and the staff operation area.

[0104] At the moisture content testing station, a close-up camera is aimed at the moisture content detector. The equipment reading extraction unit reads the current displayed value, and the current batch locking unit reads CF-YN-0826 from the sample label. After the batch identifier matches the traceability unit, the robot announces the on-site moisture content and testing time.

[0105] When a live stream user asks, "Which batch of coffee beans are being shown now?", the question semantic parsing unit identifies that the question is directly related to the current traceability fact unit. At this time, the robot is in a static shooting state, the interruption processing priority reaches the answer threshold, the system pauses the subsequent moisture content explanation and answers the current batch number. After the answer is completed, the broadcast resumes from the position of the unfinished detection time explanation.

[0106] After the robot moves to the finished product storage station, the packaging traceability code recognition result shows that the current packaging belongs to CF-YN-0827, which is inconsistent with CF-YN-0826 corresponding to the live broadcast task. The on-site consistency deviation exceeds the consistency threshold. The robot stops broadcasting about the packaging date and grading results, keeps the close-up camera at the packaging traceability code position, and prompts the staff to check the current storage location.

[0107] After the staff moved the CF-YN-0826 corresponding packaging to the shooting position, the robot reread the packaging traceability code and grading label. The on-site consistency deviation was reduced to within the consistency threshold. The system restored the packaging date and grading result broadcast and wrote the first batch inconsistency, the review time and the recovery process into the robot live broadcast execution record.

[0108] After the live broadcast ends, the robot's live broadcast execution record fully saves the arrival time of each workstation, camera footage, on-site readings, actual broadcast text, user questions, answers, on-site consistency deviations, abnormal pauses and recovery anchor points. Staff can use the robot's live broadcast execution record to verify the on-site objects and data sources corresponding to each fact of origin in the live broadcast.

[0109] Through the above processing, this system converts origin traceability data into a sequence of actions that can be executed by a mobile chassis, pan-tilt unit, camera, mechanical pointer arm, and voice broadcaster in a coffee processing base. By verifying and controlling the on-site batch and equipment readings, the system enables the live streaming robot to complete the spatial movement of the origin, display of processing stations, broadcast of traceability facts, on-site verification, and answering of user questions in the actual production environment. This avoids the problems of incorrect broadcast objects, expired on-site values, and unrecoverable actions after Q&A that occur with fixed-script robots.

Claims

1. A live-streaming marketing robot for agricultural products based on origin traceability data mining, characterized in that, It includes a registration module for workspaces at the place of origin, a compilation module for traceability fact units, a robot demonstration motion planning module, a multimodal live streaming synchronization control module, a broadcast physical consistency verification module, and an interaction interruption recovery and recording module; The origin operation space registration module performs spatial registration of workstations, agricultural product batches, evidence carriers, and robot-observable locations in the origin operation site, generating an origin operation space registration data package containing workstation coordinates, workstation access boundaries, evidence carrier locations, robot-observable locations, and robot positioning references. The traceability fact unit compilation module receives the origin operation space registration data package and origin traceability record, binds the origin traceability record with the corresponding operation station, agricultural product batch and evidence carrier, calculates the fact executability and generates an executable traceability fact unit set; The robot demonstration motion planning module receives the origin workspace registration data package and the set of executable traceability fact units, calculates the passage cost between workstations, and arranges movement actions, framing actions, evidence reading actions, mechanical pointing actions and voice broadcasting actions according to the work time sequence and process connection relationship to generate a traceability fact display task sequence. The multimodal live streaming synchronization control module receives the task sequence for tracing the facts and displays them, controls the robot to perform corresponding movement actions, framing actions, evidence reading actions, mechanical pointing actions, and voice broadcasting actions, calculates the visual quality of the screen and the multimodal synchronization deviation, and generates a multimodal display execution data package. The broadcast physical consistency verification module receives the multimodal display execution data packet and the set of executable traceability fact units. It compares the agricultural product batch identifier, evidence carrier identifier, current workstation and display time obtained at the live broadcast site with the current executable traceability fact units, calculates the on-site consistency deviation, generates consistency verification results and display correction instructions, and feeds back the display correction instructions to the robot display motion planning module and the multimodal live broadcast synchronization control module. The Interaction Interruption Recovery and Recording Module receives live interactive questions, the task sequence for tracing the facts, the consistency verification results, and the multimodal display execution data packet. It calculates the priority of handling user question interruptions, saves task breakpoints, and re-executes workstation positioning, evidence carrier reading, and physical consistency verification after the live interactive questions are handled. Based on the regenerated consistency verification results, it restores the task sequence for tracing the facts and generates a live marketing execution record.

2. The agricultural product live-streaming marketing robot based on origin traceability data mining as described in claim 1, characterized in that, The traceability fact unit compilation module includes a fact evidence reading unit, a spatiotemporal window binding unit, a field carrier binding unit, a fact executability calculation unit, and a fact unit output unit. The fact evidence reading unit reads the agricultural product batch identifier, operation type, operation time, operation position, evidence summary, and evidence carrier identifier from the origin traceability record. The spatiotemporal window binding unit determines the corresponding field display time window based on the operation time and operation position. The field carrier binding unit binds the evidence carrier identifier with the evidence carrier location and the robot's observation position based on the origin operation space registration data package. The fact executability calculation unit calculates the fact executability based on the evidence carrier's readable status, the operation position's reachability status, the origin traceability record's validity status, and the agricultural product batch consistency status. The fact unit output unit retains the origin traceability records whose fact executability reaches the execution threshold and generates an executable traceability fact unit set.

3. The agricultural product live-streaming marketing robot based on origin traceability data mining according to claim 2, characterized in that, The robot demonstration motion planning module includes a task constraint extraction unit, a passage cost calculation unit, a demonstration motion configuration unit, a motion conflict resolution unit, and a task sequence output unit. The task constraint extraction unit extracts the operation time sequence, process connection relationship, workstation, and on-site demonstration time window from the set of executable traceable fact units. The passage cost calculation unit calculates the passage cost between workstations based on the path length between workstations, obstacle spacing, robot turning times, and demonstration preparation time. The demonstration motion configuration unit configures a demonstration motion segment for each executable traceable fact unit, consisting of the target workstation, robot observation position, camera posture, mechanical pointing position, evidence reading time, and voice broadcast time period. The motion conflict resolution unit eliminates workstation occupancy conflicts and execution time conflicts between adjacent demonstration motion segments. The task sequence output unit arranges the demonstration motion segments according to the passage cost between workstations and the operation time sequence, generating a traceability fact demonstration task sequence.

4. The agricultural product live-streaming marketing robot based on origin traceability data mining as described in claim 3, characterized in that, The multimodal live streaming synchronization control module includes a mobile base control unit, a lifting gimbal control unit, a live streaming view control unit, a mechanical pointing control unit, a voice broadcast control unit, and a synchronization deviation correction unit. The mobile base control unit and the lifting gimbal control unit move to the robot's observation position according to the displayed action segment and adjust the camera posture. The live streaming view control unit calculates the visual quality of the image based on the proportion of the agricultural product in the image, the clarity of the evidence carrier, the degree of target occlusion, and the degree of motion blur. The mechanical pointing control unit controls the mechanical pointing arm to point to the agricultural product or evidence carrier corresponding to the currently executable traceability fact unit. The synchronization deviation correction unit calculates the multimodal synchronization deviation based on the image stabilization time, the mechanical pointing arrival time, and the voice broadcast start time, and reduces the multimodal synchronization deviation by delaying the voice broadcast start time, adjusting the mechanical pointing arrival time, or re-executing the live streaming view.

5. The agricultural product live-streaming marketing robot based on origin traceability data mining according to claim 4, characterized in that, The physical consistency verification module includes a field target extraction unit, an evidence carrier rereading unit, a current workstation verification unit, a fact field comparison unit, and a correction instruction output unit. The field target extraction unit extracts the physical characteristics of agricultural products from the live broadcast image corresponding to the multimodal display execution data package. The evidence carrier rereading unit reads the current evidence carrier identifier and the current agricultural product batch identifier. The current workstation verification unit determines the current workstation based on the robot positioning reference and workstation coordinates. The fact field comparison unit compares the current evidence carrier identifier, the current agricultural product batch identifier, the current workstation, and the display time with the current executable traceability fact unit to calculate the field consistency deviation. The correction instruction output unit generates a display correction instruction based on the field consistency deviation, which includes the target workstation, the robot's camera observation position, the evidence reading action, and the voice broadcast status.

6. The agricultural product live-streaming marketing robot based on origin traceability data mining according to claim 5, characterized in that, The interactive interruption recovery and recording module includes a problem association calculation unit, an interruption priority calculation unit, a task breakpoint saving unit, a problem response task insertion unit, a task recovery verification unit, and an execution record generation unit. The problem association calculation unit determines the degree of association between the live interactive problem and the currently executable traceable fact unit; The interruption priority calculation unit calculates the user's problem interruption handling priority based on the degree of correlation, the number of times the problem is repeated, the timeliness of the problem, and the current action safety status. The task breakpoint saving unit saves the current display action segment, current workstation, current agricultural product batch identifier, current evidence carrier identifier, and the position of completed voice broadcast; the problem response task insertion unit converts live interactive problems that reach the interruption handling threshold into problem response tasks and inserts them into the traceability fact display task sequence; After the problem response task ends, the task recovery verification unit re-acquires the current workstation, the current agricultural product batch identifier, and the current evidence carrier identifier, and calls the broadcast physical consistency verification module to restore the task breakpoint.

7. The agricultural product live-streaming marketing robot based on origin traceability data mining as described in claim 6, characterized in that, The consistency verification results include a consistent state, a correctable deviation state, and a prohibition on continuing broadcasting state. A consistent state is generated when the on-site consistency deviation does not reach the first correction threshold. A correctable deviation state is generated when the on-site consistency deviation reaches the first correction threshold but does not reach the second abort threshold, and a display correction instruction is executed. A prohibition on continuing broadcasting state is generated when the on-site consistency deviation reaches the second abort threshold or when the current agricultural product batch identifier is inconsistent with the agricultural product batch identifier in the current executable traceability fact unit. The current voice broadcasting is stopped and the workstation positioning and evidence carrier reading are re-executed.

8. The agricultural product live-streaming marketing robot based on origin traceability data mining according to claim 7, characterized in that, The execution record generation unit writes the version of the traceability fact display task sequence, the identifier of the executable traceability fact unit, the current workstation, the current agricultural product batch identifier, the current evidence carrier identifier, the live broadcast frame index, the voice broadcast start time, the mechanical pointing arrival time, the consistency verification result, the live interactive issues, the task breakpoint and the task recovery time into the live marketing execution record, and establishes the association between the live marketing execution record and the origin traceability record according to the identifier of the executable traceability fact unit.