Farmer market transaction behavior identification method and system combined with intelligent electronic scale

Abstract

The invention discloses a farmer market transaction behavior identification method and system combined with an intelligent electronic scale, and relates to the technical field of market transaction management, and the method comprises the steps: determining a first commercial tenant region of a farmer market, recording a continuous weighing data sequence through the intelligent electronic scale, tracking commercial tenants through an industrial following camera, and generating an image monitoring sequence; positioning a weighing stable time zone, calculating stable weight, and generating an accurate stable weighing result; performing analysis based on the image monitoring sequence to generate a first abnormal behavior recognition result; comparing the stable weighing result with the payment result, carrying out transaction abnormity identification, and generating a second abnormal behavior identification result; and performing dual abnormity verification by combining the two identification results, and triggering a transaction abnormity alarm according to the verification result. The technical problems that in the prior art, merchant transaction behaviors cannot be comprehensively monitored, and the capability of rapidly recognizing and processing abnormal transactions is lacked are solved, and the technical effect of improving transparency and safety of the farmer's market transaction process is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of market transaction management, and in particular to a method and system for identifying transaction behaviors in a farmer's market in combination with an intelligent electronic scale. Background Art

[0002] With the expansion of farmers' markets and the increasing complexity of trading behaviors, the transparency and security of the transaction process have become particularly important. However, the traditional farmers' market trading method still mainly relies on simple electronic scales to record weighing data, which makes it difficult to fully monitor the merchant's operating behavior and transaction process. Especially after weighing is completed, merchants may replace or remove some goods, and there is a lack of intuitive verification of whether the user's payment amount matches the actual weighing result, which can easily lead to transaction disputes. Traditional transaction monitoring methods cannot fully monitor merchant transaction behaviors, lack the ability to quickly identify and handle abnormal transactions, and are difficult to meet the modern farmers' market's needs for efficient management and real-time monitoring of transaction behaviors. Summary of the invention

[0003] The present application provides a method and system for identifying transaction behavior in a farmers' market in combination with a smart electronic scale, which is used to solve the technical problems that the existing technology cannot comprehensively monitor the transaction behavior of merchants and lacks the ability to quickly identify and process abnormal transactions.

[0004] In view of the above problems, the present application provides a method and system for identifying transaction behaviors in farmers' markets in combination with a smart electronic scale.

[0005] In a first aspect of the present application, a method for identifying transaction behavior in a farmers' market in combination with a smart electronic scale is provided, the method comprising: A first area of ​​a first merchant in a farmer's market is determined, wherein a smart electronic scale and an industrial follow-up camera are provided in the first area, and the industrial follow-up camera follows the first merchant for monitoring; when the smart electronic scale is activated, a continuous weighing data sequence is recorded, and at the same time, the first merchant is tracked by the industrial follow-up camera to obtain an image monitoring sequence; based on the image monitoring sequence, the weighing behavior of the first merchant is identified, and a weighing stable time zone is located; according to the weighing stable time zone, a stable weight calculation is performed on the continuous weighing data sequence to generate a stable weighing result; based on the image monitoring sequence, a transaction behavior between the first merchant and a user is identified, and a first abnormal behavior identification result is generated; the transaction payment platform of the first merchant is connected to obtain a payment result, and a transaction abnormality identification is performed based on the stable weighing result and the payment result to generate a second abnormal behavior identification result; double abnormality verification is performed with the first abnormal behavior identification result and the second abnormal behavior identification result, and a transaction abnormality alarm is issued according to the verification result.

[0006] The second aspect of the present application provides a farmer's market transaction behavior identification system combined with a smart electronic scale, the system comprising: An area determination module, the area determination module is used to determine a first area of ​​a first merchant in the farmer's market, wherein a smart electronic scale and an industrial follow-up camera are arranged in the first area, and the industrial follow-up camera follows the first merchant for monitoring; an image monitoring module, the image monitoring module is used to record a continuous weighing data sequence when the smart electronic scale is activated, and simultaneously track the first merchant through the industrial follow-up camera to obtain an image monitoring sequence; a weighing behavior recognition module, the weighing behavior recognition module is used to identify the weighing behavior of the first merchant based on the image monitoring sequence and locate the weighing stable time zone; a stable weight calculation module, the stable weight calculation module is used to calculate the continuous weighing data sequence according to the weighing stable time zone. A weighing data sequence is used to perform stable weight calculation to generate a stable weighing result; a first transaction behavior identification module is used to identify the transaction behavior between the first merchant and the user based on the image monitoring sequence to generate a first abnormal behavior identification result; a second transaction behavior identification module is used to connect to the transaction payment platform of the first merchant to obtain the payment result, perform transaction anomaly identification based on the stable weighing result and the payment result, and generate a second abnormal behavior identification result; a double abnormal verification module is used to perform double abnormal verification with the first abnormal behavior identification result and the second abnormal behavior identification result, and issue a transaction abnormality alarm according to the verification result.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: The present application determines a first area of ​​a first merchant in a farmer's market, wherein a smart electronic scale and an industrial follow-up camera are provided in the first area, and the industrial follow-up camera follows the first merchant for monitoring; when the smart electronic scale is activated, a continuous weighing data sequence is recorded, and at the same time, the first merchant is tracked by the industrial follow-up camera to obtain an image monitoring sequence; based on the image monitoring sequence, the weighing behavior of the first merchant is identified, and a stable weighing time zone is located; according to the stable weighing time zone, a stable weight calculation is performed on the continuous weighing data sequence to generate a stable weighing result; based on the image monitoring sequence, the transaction behavior between the first merchant and the user is identified, and a first abnormal behavior identification result is generated; the transaction payment platform of the first merchant is connected to obtain a payment result, and a transaction anomaly identification is performed based on the stable weighing result and the payment result to generate a second abnormal behavior identification result; double abnormality verification is performed with the first abnormal behavior identification result and the second abnormal behavior identification result, and a transaction abnormality alarm is issued according to the verification result. The present invention solves the technical problems that the prior art cannot comprehensively monitor merchant transaction behaviors and lacks the ability to quickly identify and process abnormal transactions. By combining intelligent electronic scales and industrial follow-up cameras, accurate identification of merchant weighing and transaction behaviors is achieved through weighing data and image monitoring sequence analysis. By using technologies such as stable weight calculation, payment result comparison, and double abnormality verification, transaction anomalies can be quickly discovered and alarms can be issued, thereby achieving the technical effect of improving the transparency and security of the transaction process in the farmers' market. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0009] Figure 1 A schematic diagram of a method for identifying transaction behaviors in a farmers' market in combination with a smart electronic scale provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a farmer's market transaction behavior identification system combined with a smart electronic scale provided in an embodiment of the present application.

[0010] Explanation of the accompanying drawings: area determination module 11, image monitoring module 12, weighing behavior recognition module 13, stable weight calculation module 14, first transaction behavior recognition module 15, second transaction behavior recognition module 16, double abnormality verification module 17. DETAILED DESCRIPTION

[0011] The present application provides a method and system for identifying transaction behaviors in farmers' markets in combination with smart electronic scales, aiming to solve the technical problems that the existing technology cannot comprehensively monitor the transaction behaviors of merchants and lacks the ability to quickly identify and process abnormal transactions. By combining smart electronic scales and industrial follow-up cameras, accurate identification of merchant weighing behaviors and transaction behaviors is achieved through weighing data and image monitoring sequence analysis. Through stable weight calculation, payment result comparison, double abnormality verification and other technologies, transaction anomalies can be quickly discovered and alarms can be issued, thereby achieving the technical effect of improving the transparency and security of the transaction process in farmers' markets.

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0013] It should be noted that any variations of the terms "include" and "have" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0014] Embodiment 1, as Figure 1 As shown, the present application provides a method for identifying transaction behavior in a farmers' market in combination with a smart electronic scale, the method comprising: Step S100: determining a first area of ​​a first merchant in a farmer's market, wherein a smart electronic scale and an industrial follow-up camera are arranged in the first area, and the industrial follow-up camera follows the first merchant for monitoring.

[0015] In the embodiments of the present application, when data collection is involved, such as obtaining data through electronic scales and cameras, user permission or consent is required, and the collection, use and processing of relevant data comply with relevant laws, regulations and standards of relevant countries and regions, and will not violate the public interest or infringe on personal privacy. In the monitoring of trading behavior in the farmers' market, the first area of ​​the first merchant is first determined. The first area refers to the monitoring range defined around the stall of the first merchant, which is used for centralized management of weighing and trading behaviors. The area is pre-set manually.

[0016] Smart electronic scales and industrial follow-up cameras are installed in the first area. The smart electronic scales are used to collect the weighing data of merchants. At the same time, industrial follow-up cameras are installed in the first area. The cameras have high resolution and dynamic tracking capabilities. Through the built-in target recognition algorithm, they can lock and follow the behavior trajectory of the first merchant in real time. The cameras capture the dynamic behavior of merchants and generate complete image sequence data for behavior recognition and verification.

[0017] Step S200: When the smart electronic scale is activated, a continuous weighing data sequence is recorded, and at the same time, the first merchant is tracked by the industrial follow-up camera to obtain an image monitoring sequence.

[0018] In the embodiment of the present application, when the smart electronic scale is activated, the weighing data collection and the merchant behavior monitoring are started at the same time. The smart electronic scale records the continuous weighing data sequence in real time and captures the weight change process. At the same time, the industrial follow-up camera tracks the first merchant through the pre-trained target detection model and collects its behavior images.

[0019] Specifically, the industrial tracking camera uses the target detection model to identify and lock the dynamic features of the merchant to ensure accurate tracking. By analyzing the merchant's displacement characteristics in continuous image frames, the merchant's target position is predicted, and the camera's control parameters (such as focal length, angle, etc.) are dynamically adjusted to keep the merchant within the monitoring range. Finally, a complete image sequence of the merchant's behavior is recorded to generate an image monitoring sequence.

[0020] Furthermore, in the method provided in the embodiment of the application, the first merchant is tracked by the industrial follow-up camera to obtain an image monitoring sequence, and further includes: A first target detection model is configured for the first merchant; images are captured in the first area by the industrial follow-up camera, and the first target detection model is called to perform target detection, so as to establish a first image set and a first camera control parameter; based on the first image set, the inter-frame dynamic features of the first merchant are analyzed, target displacement is predicted, and a predicted displacement is generated; with the predicted displacement as a reference, the first camera control parameter is adjusted to complete the tracking of the first merchant and generate the image monitoring sequence.

[0021] In the embodiment of the present application, a first target detection model is first configured, which is trained based on the labeled data of the appearance and behavior characteristics of the first merchant obtained from the historical database, and adopts the YOLO target detection algorithm to enable the model to accurately identify the dynamic characteristics of the first merchant. After the training is completed, the first target detection model is deployed in the camera system for real-time detection of the first merchant.

[0022] In the first area, the industrial follow-up camera starts wide-area image acquisition and performs target detection by calling the first target detection model frame by frame. Since the first merchant is not necessarily included in the initial image frame, the merchant target is successfully detected by analyzing the continuous image frames. After the first merchant is detected, its initial position is recorded and marked as the core target for subsequent tracking. Subsequently, the industrial follow-up camera continues to collect image frames containing the first merchant to form a first image set. This image set stores the continuous dynamic features of the merchant and establishes the first camera control parameters, including focal length, rotation angle, and viewing angle range, which are used to adjust the camera in real time to adapt to the movement of the merchant.

[0023] Next, the inter-frame dynamic features of the first merchant are analyzed based on the first image set. This process is completed by the optical flow method, which calculates the motion vectors of the merchant's key points in adjacent image frames, extracts its moving direction, speed and position information, and clusters the dynamic information of these key points to form the merchant's overall motion trajectory. Subsequently, the Kalman filter algorithm is used to predict the merchant's future position. The Kalman filter model predicts the merchant's possible position in the next frame by constructing a state vector, including the merchant's current coordinates, speed and acceleration. By smoothing the noise of the input data, the Kalman filter model outputs predicted displacement data.

[0024] The camera's control parameters are then dynamically adjusted based on the predicted displacement. Through the servo control system, the camera adjusts its rotation angle in real time based on the predicted data to ensure that the merchant is always in the center of the monitoring screen. At the same time, the focal length is dynamically adjusted based on the distance between the merchant and the camera to ensure that the picture is clear and the merchant is always presented in an appropriate proportion. This real-time tracking process allows the camera to smoothly follow the merchant's movement trajectory to avoid the target being lost or out of the monitoring range.

[0025] While the camera continuously collects images, all captured image frames are integrated into a complete image monitoring sequence. A timestamp is added to each frame to ensure time synchronization. The resulting image monitoring sequence fully records the behavior trajectory of the first merchant, providing high-quality basic data for subsequent transaction behavior analysis and anomaly detection.

[0026] Furthermore, in the method provided in the embodiment of the application, after generating the image monitoring sequence, the method further includes: Identify the tracking object of any frame image in the image monitoring sequence; if the tracking object includes other persons except the first merchant, perform blur processing on the other persons.

[0027] In the embodiment of the present application, firstly, an arbitrary frame of image is extracted from the image monitoring sequence, and the deployed target detection model is called to detect and classify the target in the image. The target detection model can identify all objects appearing in the frame, including the first merchant and other possible unrelated persons. The model prioritizes the first merchant as the main target mark through the appearance features and behavioral features annotated in the training data, and classifies other persons as "non-merchant objects".

[0028] After the target recognition is completed, all detected objects in the frame are classified. The tracked object refers to the target tracked by the camera in the current frame, that is, the first merchant. For other detected persons (such as passing customers or unrelated persons in the background), these objects are marked as other persons and need further processing to protect privacy.

[0029] To ensure that the privacy of irrelevant persons is not leaked, other detected persons are blurred. Specifically, the facial areas or the entire body contours of these persons are blurred through image processing techniques (such as Gaussian blur algorithm or pixelation).

[0030] After the blurring process is completed, the processed image frames are reinserted into the image monitoring sequence.

[0031] Step S300: identifying the weighing behavior of the first merchant based on the image monitoring sequence, and locating the weighing stable time zone.

[0032] In an embodiment of the present application, based on the image monitoring sequence, by identifying the weighing behavior of the first merchant and combining the weighing cutoff behavior characteristics, a key image frame sequence from the merchant completing the operation of adding or adjusting items to finally removing the items is extracted, the corresponding acquisition time is obtained, and combined with the fluctuation trend of the weighing data, the time interval when the weighing data is stable is located, and finally the weighing stable time zone is determined.

[0033] Furthermore, in the method provided in the embodiment of the application, the weighing behavior of the first merchant is identified based on the image monitoring sequence, and the weighing stable time zone is located, and the method further includes: Configure a weighing cutoff behavior feature, wherein the weighing cutoff behavior feature includes a continuous behavior feature of a time zone; identify the image monitoring sequence based on the weighing cutoff behavior feature, and locate the weighing image frame sequence corresponding to the weighing cutoff behavior feature; obtain the acquisition time corresponding to the weighing image frame sequence, and establish the weighing stable time zone.

[0034] In the embodiment of the present application, the weighing cut-off behavior characteristics are first configured, and the action characteristics of the merchant during the weighing process are defined by rules. The weighing cut-off behavior characteristics include a continuous behavior pattern within a time zone, such as the merchant's adjustment actions of adding or removing items multiple times during the weighing process, the state of the items when they are stationary, and the termination action of the items being removed from the electronic scale. Through manually set rules, such as the activity characteristics of the hand in the electronic scale area, the adjustment frequency, and the stationary time threshold, the standard of the weighing cut-off behavior is formed for the subsequent identification process.

[0035] Next, the image monitoring sequence is identified based on the weighing cutoff behavior characteristics. By analyzing the image monitoring sequence frame by frame, the frame difference method and background modeling technology are used to detect and extract the merchant's actions. The frame difference method calculates the pixel differences between adjacent frames to identify dynamic areas and determine whether the merchant's hands or items move near the electronic scale. Background modeling builds a dynamic background model to eliminate static background areas and only retain dynamic changes in the foreground. The motion features detected in continuous image frames are counted. When it is detected that the merchant's adjustment action is terminated (such as the hand is no longer close to the electronic scale area for more than 3 seconds) and the item has not moved further, these frames are marked as weighing end frames, and a weighing image frame sequence covering the entire weighing process is generated.

[0036] Then, the timestamp of each frame in the weighing image frame sequence is extracted and time-synchronized with the weight data of the electronic scale. The weight data curve is calculated by window sliding average (window size is 1 second), and the weight fluctuation amplitude (such as standard deviation or range) is calculated in each time window. When the weight fluctuation amplitude is less than the set threshold (such as the standard deviation is less than 0.02kg) for 3 consecutive seconds, the weight is judged to have stabilized, and the time period is marked as the weight stability interval. At the same time, the merchant behavior in the image frame is verified to ensure that the weight stability interval corresponds to the merchant's static behavior (such as the hand leaving the electronic scale area for more than 3 seconds and not returning).

[0037] Finally, by combining weight fluctuation analysis and static motion verification, the time period when the weight data is stable and the merchant operation is completed is marked, which is the weighing stable time zone.

[0038] Step S400: performing stable weight calculation on the continuous weighing data sequence according to the weighing stable time zone to generate a stable weighing result.

[0039] In the embodiment of the present application, according to the weighing stable time zone, the target data of the corresponding time period is extracted from the continuous weighing data sequence, that is, the target continuous weighing data sequence. Subsequently, the extracted data is subjected to fluctuation identification, the fluctuation amplitude of the weight data is analyzed, and the time period with the smallest fluctuation amplitude is determined by the sliding window method. Finally, the data within the stable time period is averaged to generate the final stable weighing result.

[0040] Furthermore, in the method provided in the embodiment of the application, according to the weighing stable time zone, the stable weight calculation is performed on the continuous weighing data sequence to generate a stable weighing result, and further includes: According to the weighing stable time zone, a target continuous weighing data sequence corresponding to the time zone is extracted from the continuous weighing data sequence; fluctuations are identified on the target continuous weighing data sequence, and average weighing data corresponding to the time zone with the smallest continuous fluctuation amplitude is obtained to generate the stable weighing result.

[0041] In the embodiment of the present application, firstly, based on the calibrated weighing stable time zone, a data subset corresponding to the time period is extracted from the continuous weighing data sequence, which is called the target continuous weighing data sequence. The weighing stable time zone is the key time period in which the weight data fluctuates the least and reflects the real weight of the object. Through timestamp synchronization, data points matching the stable time zone are intercepted from the continuous weighing data sequence to obtain the target continuous weighing data sequence.

[0042] Next, the target continuous weighing data sequence is subjected to fluctuation identification. The data sequence is traversed step by step through the sliding window method (for example, the window size is 1 second), the fluctuation range of the weight data in each window is calculated, and statistical indicators such as standard deviation or range are used for quantification. When the fluctuation range of multiple continuous windows is less than the preset threshold (for example, the standard deviation is less than 0.02kg), the time period is determined to be the most stable interval of the weight data.

[0043] Then, the average weighing data is calculated for the identified minimum fluctuation period. In this period, all weight data points are summed and averaged to generate the final stable weight value. Finally, the calculated stable weight value is output as the stable weighing result.

[0044] Step S500: Identify the transaction behavior between the first merchant and the user based on the image monitoring sequence, and generate a first abnormal behavior identification result.

[0045] In an embodiment of the present application, in order to generate the first abnormal behavior recognition result, based on the image monitoring sequence, by configuring the transaction cut-off behavior features and the transaction abnormality features, combined with the end time of the weighing stable time zone, the transaction behavior between the first merchant and the user is analyzed. Taking the end point of the weighing stable time zone as the recognition starting point and the transaction cut-off behavior features as the recognition end point, the image monitoring sequence is frame segmented to generate a segmented image frame sequence. After that, the transaction abnormality features are further used to collect abnormal samples, train and construct the first abnormal recognition network. The segmented image frame sequence is input into the network for analysis to identify potential abnormal behaviors in the transaction process, such as whether the merchant has changed the product after the weighing is completed, and finally generate the first abnormal behavior recognition result.

[0046] Furthermore, in the method provided in the embodiment of the application, identifying the transaction behavior between the first merchant and the user based on the image monitoring sequence and generating a first abnormal behavior identification result also includes: Configure transaction cut-off behavior features and transaction abnormality features; use the end point time corresponding to the weighing stable time zone as the identification starting point, use the transaction cut-off behavior features as the identification end point, perform image frame segmentation on the image monitoring sequence, and generate a segmented image frame sequence; collect abnormal recognition samples based on the transaction abnormality features, train and construct a first abnormal recognition network; input the segmented image frame sequence into the first abnormal recognition network for analysis, and generate the first abnormal behavior recognition result.

[0047] In the embodiment of the present application, the transaction end behavior feature and the transaction abnormality feature are first configured, wherein the transaction end behavior feature is used to describe the standard action of the merchant to complete the transaction, such as handing over the goods to the user after weighing or confirming the completion of the transaction, etc.; the transaction abnormality feature is used to mark potential non-compliant behaviors, such as abnormal operations such as replacing goods after weighing, partially removing goods, or not delivering goods. By annotating the historical transaction video data, the key behavior action features of the merchant are extracted to form a transaction end behavior feature library and a transaction abnormality feature library.

[0048] Then, starting from the end time of the weighing stable time zone, the image monitoring sequence is parsed frame by frame through time synchronization analysis, and the transaction completion action in the transaction end behavior feature is taken as the end point to extract the relevant frame sequence of the transaction behavior. By combining timestamp and frame segmentation technology, the frame difference method and dynamic target detection technology are used to separate the interactive action frames between merchants and users, and generate a segmented image frame sequence covering the entire transaction process.

[0049] Then, based on the transaction abnormality features, anomaly identification samples are collected and the first anomaly identification network is trained. By simulating abnormal transaction scenarios (such as replacing goods after weighing, taking out some goods from goods, etc.), a labeled abnormal behavior sample set is generated, and the merchant's actions are analyzed using action feature extraction technology (such as HOG features and key point detection), and these features are input into the classification model (such as SVM, random forest or shallow neural network) for training. The trained first anomaly identification network focuses on the abnormal action detection of merchants and has the ability to identify complex abnormal behaviors.

[0050] Finally, the segmented image frame sequence is input into the first anomaly recognition network frame by frame for analysis. The network extracts the merchant's action features and compares them with the abnormal feature library to detect whether there is abnormal behavior. For example, whether the merchant changes the weighing of goods or partially removes goods. For the detected abnormal behavior, its type is marked, and the corresponding timestamp and image frame information are recorded, and finally the first abnormal behavior recognition result is generated, including the description of the abnormal behavior, the time of occurrence, and the corresponding image evidence.

[0051] Furthermore, the method provided in the application embodiment also includes: The segmented image frame sequence is identified to locate the weighing target features and the set of occluded image frames in which the weighing target features are occluded; the temporal continuity between the image frames in the set of occluded image frames is analyzed to identify continuous occlusion intervals; a reliability analysis is performed on the first abnormal behavior recognition result based on the continuous occlusion intervals to generate a first reliability index; and the first abnormal behavior recognition result is optimized based on the first reliability index.

[0052] In an embodiment of the present application, the segmented image frame sequence is first identified to locate the weighing target features. Specifically, a target detection algorithm (such as YOLOv4) is used to detect the weighing commodity features in each frame of the image through a trained model. The training process of the model is to first extract the image data of the weighing commodity from the historical monitoring video, manually mark the bounding box and category of the commodity, and form a training data set. Through multiple rounds of iterative training, the performance of the model's bounding box regression, classification, and confidence prediction is optimized. The trained model can generate the bounding box, detection confidence, and position features of the weighing commodity in each frame of the image. The monitoring confidence is a quantitative value of the model's credibility of the target detection result (ranging from 0-1). Through detection, a feature set of the weighing target in each frame of the image is obtained, including position, size, and confidence.

[0053] Next, detect the occlusion of the weighing target in the image and generate a set of occluded image frames. Specifically, occlusion judgment is performed based on the confidence of target detection and the regional integrity of the bounding box. When the confidence of a frame is lower than the preset threshold (such as 0.5), the frame is marked as a possible occlusion frame; if the area of ​​the bounding box is reduced by more than 50% compared with the previous frame, it is further confirmed that the target feature is partially occluded. All image frames that meet the occlusion conditions are classified into the set of occluded image frames, and their corresponding timestamps are recorded. Through this step, a set containing all occluded frames is generated, and the occlusion information of each frame is annotated.

[0054] Then, the timestamps in the set of occluded image frames are analyzed to identify the continuous intervals of occlusion. The occluded image frames are arranged in chronological order by the timestamp interval method, and the time intervals between adjacent occluded frames are calculated. If the interval is less than the set value (such as 1 second), these frames are regarded as continuous occlusions; the start time, end time and duration of the occlusion are counted, and the time range and length of each occlusion are marked. Through this step, all continuous occlusion intervals are identified.

[0055] Then, based on the continuous occlusion interval, the reliability analysis of the first abnormal behavior recognition result is performed to generate the first reliability index. The reliability analysis combines the duration of the occlusion and the degree of target feature loss. By counting the proportion of the continuous occlusion time to the total segmented image frame sequence time, recorded as P, and calculating the degree of feature loss in the occluded frame (such as the proportion of the bounding box area reduction), that is, F. By calculating the difference between 1 and P, and calculating the difference between 1 and F, the two calculated differences are weighted, where the two differences have the same weight, and the first reliability index (range 0~1) is obtained by calculation.

[0056] Finally, the first reliability index is used to optimize the abnormal behavior recognition results. If the reliability is low (such as the first reliability index is lower than 0.6), the re-detection process is triggered, the occluded frames are ignored, and only the unoccluded frames are re-detected for the completeness and consistency of the key actions of the abnormal behavior. Specifically, all frames marked as occluded are first removed from the segmented image frame sequence, and only the unoccluded frames are retained for subsequent analysis. Then, in the unoccluded frames, the key actions of the abnormal behavior are re-detected, such as whether the merchant changes the weighing goods or partially removes the goods. The action analysis algorithm (such as frame difference method or key point tracking) is used to extract the hand movements of the merchant and the change trajectory of the weighing target to verify the occurrence of abnormal behavior. Finally, combined with the action feature sequence of the unoccluded frame, it is verified whether the features of the abnormal behavior are complete and consistent. For example, check whether the change process of the weighing target is logical (such as the reduction in mass corresponding to the removal of goods), and verify whether the time series of abnormal features is continuous. If the abnormal feature sequence can be reasonably compensated in the unoccluded frame, it is considered complete; if the key feature loss caused by occlusion cannot be compensated, it is marked as a false alarm to exclude the wrong result.

[0057] For results with reliability higher than the set threshold (e.g. 0.6), the abnormal behavior recognition results are directly confirmed and the reliability score is added as output information. The optimized results include the type of abnormal behavior, the time of occurrence, the relevant image frames and their reliability indicators.

[0058] Step S600: connecting to the transaction payment platform of the first merchant to obtain the payment result, performing transaction anomaly identification based on the stable weighing result and the payment result, and generating a second abnormal behavior identification result.

[0059] In the embodiment of the present application, the transaction payment platform of the first merchant is first connected to obtain the payment result, and the second abnormal behavior identification result is generated based on the comparison between the stable weighing result and the payment result. The transaction payment platform is a system for merchants and users to complete payment transactions, which records key information such as the unit price, payment amount, and payment weight of the goods. Real-time data connection is established with the payment platform through a secure API interface, and the merchant's transaction records, including product information, payment amount, and payment weight, are extracted from the payment platform.

[0060] After obtaining the payment result, the transaction weight corresponding to the payment result is calculated using the payment amount and the unit price of the goods. The transaction weight is calculated by dividing the unit price of the goods by the payment amount.

[0061] Next, the calculated transaction weight is directly compared with the stable weighing result to determine whether the two are consistent. If the deviation between the two exceeds the set tolerance threshold (such as 3%), the transaction is marked as abnormal and a second abnormal behavior identification result is generated. The second abnormal behavior identification result is used to determine whether the merchant calculates the fee according to the stable weighing result and identify possible abnormal behavior. For example, if the weight corresponding to the payment result is higher than the stable weighing result, it is marked as "overcharged", indicating that the merchant charges too much or has malicious behavior.

[0062] Step S700: performing double abnormality verification using the first abnormal behavior recognition result and the second abnormal behavior recognition result, and issuing a transaction abnormality alarm based on the verification results.

[0063] In the embodiment of the present application, when double abnormal verification is performed based on the first abnormal behavior recognition result and the second abnormal behavior recognition result, as long as any one of the results shows an abnormality, the transaction abnormality alarm is triggered. Specifically, when the first abnormal behavior recognition result shows that the merchant has abnormal operations after weighing (such as changing goods or removing some goods), or the second abnormal behavior recognition result shows that the payment behavior does not match the weighing data (such as overcharging), the transaction will be directly judged as abnormal without both being abnormal. During verification, time matching and behavior logic checks are combined to ensure accurate judgment of abnormalities.

[0064] Once an abnormality is confirmed, whether it is an abnormal weighing behavior or an abnormal payment behavior, a transaction abnormality alarm is immediately generated. The alarm information includes the type of abnormality (such as weighing change, product removal, overcharge, etc.), the time range of the abnormality, and the relevant weight deviation and fee deviation data. Through this mechanism, rapid identification and alarm are carried out to ensure the transparency and standardization of transaction behavior.

[0065] In the embodiments of the present application, in summary, the embodiments of the present application have at least the following technical effects: The present application determines a first area of ​​a first merchant in a farmer's market, wherein a smart electronic scale and an industrial follow-up camera are provided in the first area, and the industrial follow-up camera follows the first merchant for monitoring; when the smart electronic scale is activated, a continuous weighing data sequence is recorded, and at the same time, the first merchant is tracked by the industrial follow-up camera to obtain an image monitoring sequence; based on the image monitoring sequence, the weighing behavior of the first merchant is identified, and a stable weighing time zone is located; according to the stable weighing time zone, a stable weight calculation is performed on the continuous weighing data sequence to generate a stable weighing result; based on the image monitoring sequence, the transaction behavior between the first merchant and the user is identified, and a first abnormal behavior identification result is generated; the transaction payment platform of the first merchant is connected to obtain a payment result, and a transaction anomaly identification is performed based on the stable weighing result and the payment result to generate a second abnormal behavior identification result; double abnormality verification is performed with the first abnormal behavior identification result and the second abnormal behavior identification result, and a transaction abnormality alarm is issued according to the verification result. The present invention solves the technical problems that the prior art cannot comprehensively monitor merchant transaction behaviors and lacks the ability to quickly identify and process abnormal transactions. By combining intelligent electronic scales and industrial follow-up cameras, accurate identification of merchant weighing and transaction behaviors is achieved through weighing data and image monitoring sequence analysis. By using technologies such as stable weight calculation, payment result comparison, and double abnormality verification, transaction anomalies can be quickly discovered and alarms can be issued, thereby achieving the technical effect of improving the transparency and security of the transaction process in the farmers' market.

[0066] Embodiment 2 is based on the same inventive concept as the method for identifying transaction behavior in farmers' markets in combination with a smart electronic scale in the aforementioned embodiment. Figure 2 As shown, the present application provides a farmer's market transaction behavior identification system combined with a smart electronic scale. The system and method embodiments in the present application are based on the same inventive concept. The system includes: An area determination module 11 is used to determine a first area of ​​a first merchant in a farmer's market, wherein a smart electronic scale and an industrial follow-up camera are provided in the first area, and the industrial follow-up camera follows the first merchant for monitoring; an image monitoring module 12 is used to record a continuous weighing data sequence when the smart electronic scale is activated, and simultaneously track the first merchant through the industrial follow-up camera to obtain an image monitoring sequence; a weighing behavior recognition module 13 is used to recognize the weighing behavior of the first merchant based on the image monitoring sequence and locate the weighing stable time zone; a stable weight calculation module 14 is used to calculate the stable weight of the first merchant according to the weighing stable time zone. A continuous weighing data sequence is used to perform stable weight calculation to generate a stable weighing result; a first transaction behavior identification module 15, the first transaction behavior identification module 15 is used to identify the transaction behavior between the first merchant and the user based on the image monitoring sequence, and generate a first abnormal behavior identification result; a second transaction behavior identification module 16, the second transaction behavior identification module 16 is used to connect to the transaction payment platform of the first merchant to obtain the payment result, and perform transaction anomaly identification based on the stable weighing result and the payment result to generate a second abnormal behavior identification result; a double abnormal verification module 17, the double abnormal verification module 17 is used to perform double abnormal verification with the first abnormal behavior identification result and the second abnormal behavior identification result, and issue a transaction abnormality alarm according to the verification result.

[0067] Furthermore, the system is also used to implement the following functions: Configure a weighing cutoff behavior feature, wherein the weighing cutoff behavior feature includes a continuous behavior feature of a time zone; identify the image monitoring sequence based on the weighing cutoff behavior feature, and locate the weighing image frame sequence corresponding to the weighing cutoff behavior feature; obtain the acquisition time corresponding to the weighing image frame sequence, and establish the weighing stable time zone.

[0068] Furthermore, the system is also used to implement the following functions: Configure transaction cut-off behavior features and transaction abnormality features; use the end point time corresponding to the weighing stable time zone as the identification starting point, use the transaction cut-off behavior features as the identification end point, perform image frame segmentation on the image monitoring sequence, and generate a segmented image frame sequence; collect abnormal recognition samples based on the transaction abnormality features, train and construct a first abnormal recognition network; input the segmented image frame sequence into the first abnormal recognition network for analysis, and generate the first abnormal behavior recognition result.

[0069] Furthermore, the system is also used to implement the following functions: The segmented image frame sequence is identified to locate the weighing target features and the set of occluded image frames in which the weighing target features are occluded; the temporal continuity between the image frames in the set of occluded image frames is analyzed to identify continuous occlusion intervals; a reliability analysis is performed on the first abnormal behavior recognition result based on the continuous occlusion intervals to generate a first reliability index; and the first abnormal behavior recognition result is optimized based on the first reliability index.

[0070] Furthermore, the system is also used to implement the following functions: A first target detection model is configured for the first merchant; images are captured in the first area by the industrial follow-up camera, and the first target detection model is called to perform target detection, so as to establish a first image set and a first camera control parameter; based on the first image set, the inter-frame dynamic features of the first merchant are analyzed, target displacement is predicted, and a predicted displacement is generated; with the predicted displacement as a reference, the first camera control parameter is adjusted to complete the tracking of the first merchant and generate the image monitoring sequence.

[0071] Furthermore, the system is also used to implement the following functions: Identify the tracking object of any frame image in the image monitoring sequence; if the tracking object includes other persons except the first merchant, perform blur processing on the other persons.

[0072] Furthermore, the system is also used to implement the following functions: According to the weighing stable time zone, a target continuous weighing data sequence corresponding to the time zone is extracted from the continuous weighing data sequence; fluctuations are identified on the target continuous weighing data sequence, and average weighing data corresponding to the time zone with the smallest continuous fluctuation amplitude is obtained to generate the stable weighing result.

[0073] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order and continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0075] This specification and the drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A method for identifying transaction behaviors in farmers' markets combined with intelligent electronic scales, characterized in that: include: Determine a first area of ​​a first merchant in a farmer's market, wherein a smart electronic scale and an industrial follow-up camera are arranged in the first area, and the industrial follow-up camera follows the first merchant for monitoring; When the intelligent electronic scale is activated, a continuous weighing data sequence is recorded, and at the same time, the first merchant is tracked by the industrial follow-up camera to obtain an image monitoring sequence; identifying the weighing behavior of the first merchant based on the image monitoring sequence, and locating a weighing stable time zone; According to the weighing stable time zone, stable weight calculation is performed on the continuous weighing data sequence to generate a stable weighing result; Identify the transaction behavior between the first merchant and the user based on the image monitoring sequence, and generate a first abnormal behavior identification result; Connecting to the transaction payment platform of the first merchant to obtain a payment result, performing transaction anomaly identification based on the stable weighing result and the payment result, and generating a second abnormal behavior identification result; A double abnormality verification is performed using the first abnormal behavior recognition result and the second abnormal behavior recognition result, and an abnormal transaction alarm is issued according to the verification results.

2. The method for identifying transaction behavior in a farmers' market combined with an intelligent electronic scale according to claim 1, characterized in that: Identifying the weighing behavior of the first merchant based on the image monitoring sequence and locating the weighing stable time zone includes: Configuring a weighing cut-off behavior feature, wherein the weighing cut-off behavior feature comprises a continuous behavior feature of a time zone; Identifying the image monitoring sequence based on the weighing cutoff behavior feature, and locating the weighing image frame sequence corresponding to the weighing cutoff behavior feature; The acquisition time corresponding to the weighing image frame sequence is obtained, and the weighing stable time zone is established.

3. The method for identifying transaction behavior in a farmers' market combined with an intelligent electronic scale as claimed in claim 2, characterized in that: Identifying the transaction behavior between the first merchant and the user based on the image monitoring sequence and generating a first abnormal behavior identification result includes: Configure transaction cut-off behavior characteristics and transaction abnormality characteristics; Taking the end point time corresponding to the weighing stable time zone as the identification starting point and the transaction cut-off behavior feature as the identification end point, performing image frame segmentation on the image monitoring sequence to generate a segmented image frame sequence; Based on the transaction abnormality features, anomaly recognition samples are collected and a first anomaly recognition network is trained and constructed; The segmented image frame sequence is input into the first abnormality recognition network for analysis to generate the first abnormal behavior recognition result.

4. The method for identifying transaction behavior in a farmers' market combined with an intelligent electronic scale according to claim 3, characterized in that: Also includes: Identify the segmented image frame sequence, locate the weighing target feature, and the set of blocked image frames where the weighing target feature is blocked; Analyzing the temporal continuity between the image frames in the occluded image frame set to identify continuous occlusion intervals; Performing reliability analysis on the first abnormal behavior recognition result based on the continuous occlusion interval to generate a first reliability index; The first abnormal behavior recognition result is optimized based on the first reliability index.

5. The method for identifying transaction behavior in a farmers' market combined with an intelligent electronic scale according to claim 1, characterized in that: The first merchant is tracked by the industrial follow-up camera to obtain an image monitoring sequence, including: configuring a first object detection model for the first merchant; Using the industrial follow-up camera to capture images in the first area, calling the first target detection model to perform target detection, and establishing a first image set and a first camera control parameter; Analyzing the inter-frame dynamic features of the first merchant based on the first image set, performing target displacement prediction, and generating predicted displacement; The predicted displacement is used as a reference to adjust the control parameters of the first camera to complete the tracking of the first merchant and generate the image monitoring sequence.

6. The method for identifying transaction behavior in a farmers' market combined with an intelligent electronic scale according to claim 1, characterized in that: After generating the image monitoring sequence, the method further includes: Identify the tracking object of any frame image in the image monitoring sequence; If the tracking object includes other persons except the first merchant, the other persons are blurred.

7. The method for identifying transaction behavior in a farmers' market combined with an intelligent electronic scale according to claim 1, characterized in that: According to the weighing stable time zone, the stable weight calculation is performed on the continuous weighing data sequence to generate a stable weighing result, including: According to the weighing stable time zone, a target continuous weighing data sequence corresponding to the time zone is extracted from the continuous weighing data sequence; Fluctuation identification is performed on the target continuous weighing data sequence, and average weighing data corresponding to the time zone with the smallest continuous fluctuation amplitude is obtained to generate the stable weighing result.

8. The farmer's market transaction behavior recognition system combined with the intelligent electronic scale is characterized by: The system comprises: An area determination module, the area determination module is used to determine a first area of ​​a first merchant in the farmer's market, wherein a smart electronic scale and an industrial follow-up camera are arranged in the first area, and the industrial follow-up camera follows the first merchant for monitoring; An image monitoring module, wherein when the intelligent electronic scale is activated, the image monitoring module is used to record a continuous weighing data sequence and simultaneously track the first merchant through the industrial follow-up camera to obtain an image monitoring sequence; A weighing behavior recognition module, the weighing behavior recognition module is used to identify the weighing behavior of the first merchant based on the image monitoring sequence and locate the weighing stable time zone; A stable weight calculation module, the stable weight calculation module is used to perform stable weight calculation on the continuous weighing data sequence according to the weighing stable time zone to generate a stable weighing result; a first transaction behavior recognition module, the first transaction behavior recognition module being used to recognize the transaction behavior between the first merchant and the user based on the image monitoring sequence, and to generate a first abnormal behavior recognition result; a second transaction behavior identification module, the second transaction behavior identification module being used to connect to the transaction payment platform of the first merchant to obtain a payment result, perform transaction anomaly identification based on the stable weighing result and the payment result, and generate a second abnormal behavior identification result; A double abnormality verification module is used to perform double abnormality verification based on the first abnormal behavior recognition result and the second abnormal behavior recognition result, and issue a transaction abnormality alarm based on the verification results.

Images