Anti-interference industrial vision counting method and device based on spatio-temporal context perception

Abstract

The application provides an anti-interference industrial visual counting method and device based on space-time context perception, which comprises the following steps: acquiring real-time video stream of a monitoring area and recognizing a detection target; calculating intersection proportion of the detection target and a target area; triggering video recording to acquire a target video segment when the intersection proportion is greater than or equal to an intersection threshold; extracting instantaneous position and instantaneous category of a counting target in each video frame of the target video segment; generating a position sequence based on the instantaneous position and calculating a decision displacement direction; statistically analyzing the instantaneous category and determining a decision category; comparing the decision displacement direction and the decision category with preset effective direction and effective category to generate a detection result; and counting the number of effective results in a preset detection time to obtain a counting result. The application realizes classified and directional industrial visual counting by analyzing the motion state of the counting target in continuous time.

Description

Technical Field

[0001] This invention relates to data processing technology, and more particularly to an anti-interference industrial visual counting method and apparatus based on spatiotemporal context awareness. Background Technology

[0002] With the development of industrial automation and intelligent manufacturing technologies, industrial visual counting methods have gradually replaced traditional manual counting. Industrial visual counting methods determine the quantity of target objects by acquiring images of the industrial environment and analyzing these images, thus avoiding the risks associated with manual operations.

[0003] Existing industrial vision counting methods primarily acquire single images of the target area and analyze these images to determine the number of objects within that area. However, the movement of objects in industrial environments is complex. For example, an object may pass through the target area in a predetermined direction, or it may deviate from its predetermined direction and leave the target area after entering it. Consequently, a single image acquired instantaneously can only determine the number of objects present in the target area at the time of acquisition, and cannot determine the number of objects that meet actual production needs, leading to problems of false counting and double counting.

[0004] Therefore, how to accurately count the target objects within the target area based on their motion state has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] This invention provides an anti-interference industrial visual counting method and device based on spatiotemporal context awareness, which can accurately count target objects within a target area according to the motion state of the target object.

[0006] A first aspect of the present invention provides an interference-resistant industrial visual counting method based on spatiotemporal context awareness, comprising: S1, acquire the real-time video stream of the monitored area, identify the detection target in the real-time video stream, extract the intersection area between the detection target and the preset target area, and determine the intersection ratio of the detection target based on the intersection area; S2, when the intersection ratio is determined to be greater than or equal to the preset intersection threshold, the corresponding detection target is taken as the counting target, and the counting target is recorded in video until the preset recording termination condition is met, and the target video segment corresponding to the counting target is obtained. S3, extract the instantaneous information of the counting target in each video frame of the corresponding target video segment, and determine the detection result of the corresponding counting target based on the instantaneous information; S4, obtain the counting results within the preset detection time based on the detection results of each counting target.

[0007] Optionally, in one possible implementation of the first aspect, the step S1 of identifying the detected target in the real-time video stream, extracting the intersection region between the detected target and a preset target region, and determining the intersection ratio of the detected target based on the intersection region includes: Identify targets in a real-time video stream and generate bounding boxes for those targets; Extract the intersection region between the outer detection frame and the target region, and calculate the area of ​​the intersection region as the intersection area; The area of ​​the outer detection box is obtained as the outer area, and the intersection ratio is obtained based on the ratio of the intersection area to the outer area.

[0008] Optionally, in one possible implementation of the first aspect, step S2, which involves recording video of the counting target until a preset recording termination condition is met to obtain a target video segment corresponding to the counting target, includes: Video recording is performed on the counting target until the video recording duration reaches the preset recording duration, or when the counting target and the target area do not intersect and the duration of non-intersection reaches the preset waiting duration, the video recording stops, and the target video segment corresponding to the counting target is obtained.

[0009] Optionally, in one possible implementation of the first aspect, S3 includes: The instantaneous information includes instantaneous location and instantaneous category; A position sequence is generated based on multiple instantaneous positions of the counting target, and the decision displacement direction of the counting target is determined based on the position sequence. The instantaneous categories of the counting target are counted to obtain the number of categories for each instantaneous category. The instantaneous category with the largest number of categories is selected as the decision category for the counting target. The detection results of the corresponding counting targets are generated based on the decision displacement direction and decision category.

[0010] Optionally, in one possible implementation of the first aspect, determining the decision displacement direction of the counting target based on the position sequence includes: Multiple sampling sequences are obtained by sliding sampling of the position sequence based on a preset sampling window length; Extract the first and last instantaneous positions of each sampling sequence, calculate the local displacement vector of each sampling sequence based on the first and last instantaneous positions, and determine the local displacement direction of each sampling sequence based on the local displacement vector; The number of local displacements in each direction is counted as the local displacement count, and the local displacement direction corresponding to the largest local displacement count is selected as the decision displacement direction of the counting target.

[0011] Optionally, in one possible implementation of the first aspect, generating the detection result of the corresponding counting target based on the decision displacement direction and decision category includes: The test results include valid results and invalid results; Obtain the preset effective direction and effective category; when the decision displacement direction of the counting target is the same as the effective direction and the decision category is the same as the effective category, generate the effective result of the corresponding counting target. When the decision displacement direction of the counting target is different from the effective direction and / or the decision category is different from the effective category, an invalid result is generated for the corresponding counting target.

[0012] Optionally, in one possible implementation of the first aspect, S4 includes: The number of valid results within the preset detection time is used as the counting result.

[0013] A second aspect of the present invention provides an interference-resistant industrial visual counting device based on spatiotemporal context awareness, comprising: The identification module is used to acquire real-time video streams of the monitored area, identify the detection targets in the real-time video streams, extract the intersection area between the detection targets and the preset target area, and determine the intersection ratio of the detection targets based on the intersection area. The judgment module is used to determine that when the intersection ratio is greater than or equal to a preset intersection threshold, the corresponding detection target is used as a counting target, and the counting target is recorded on video until the preset recording termination condition is met, so as to obtain the target video segment corresponding to the counting target. The detection module is used to extract instantaneous information of the counting target in each video frame of the corresponding target video segment, and determine the detection result of the corresponding counting target based on the instantaneous information; The counting module is used to obtain the counting results within a preset detection time based on the detection results of each counting target.

[0014] A third aspect of the present invention provides an electronic device comprising: a memory, a processor, and a computer program, the computer program being stored in the memory, and the processor executing the computer program to perform the methods described in the first aspect of the present invention and various possible methods related to the first aspect.

[0015] A fourth aspect of the present invention provides a storage medium storing a computer program, which, when executed by a processor, is used to implement the first aspect of the present invention and various methods possibly involved in the first aspect.

[0016] The beneficial effects of this invention are as follows: 1. This invention identifies targets in a real-time video stream and calculates the intersection ratio. When the intersection ratio is greater than or equal to an intersection threshold, a target is identified and video recording is triggered. After triggering video recording, a target video segment is acquired, recording the complete motion process of the target. Instantaneous information from each frame of the target video segment is extracted and analyzed to obtain the corresponding counting result. Unlike existing technologies that count targets using a single image, this invention records the continuous motion process of the target to obtain a target video segment. Based on the target video segment, the motion state of the target is analyzed to distinguish the decision displacement direction and decision category of the target, achieving counting by category and direction.

[0017] 2. This invention generates a position sequence based on the instantaneous position of the target in each video frame. For the position sequence, local displacement vectors are calculated through sliding sampling, and the local displacement direction is determined based on these vectors. The local displacement direction corresponding to the largest number of local displacements is selected as the decision displacement direction for the target. Simultaneously, the instantaneous categories of the target in each video frame are statistically analyzed, and the instantaneous category with the largest number of categories is selected as the decision category. This invention statistically analyzes multiple instantaneous positions and categories within the target video segment, eliminating target direction deviations caused by image jitter and target category deviations caused by reflections or occlusions.

[0018] 3. This invention performs a dual comparison between the decision displacement direction and decision category of the counting target and the preset valid direction and valid category. When both direction and category match, a valid result is generated and included in the counting result. This invention eliminates interfering targets that match the category but not the direction, or that do not match the category, through category-based and direction-based detection. Attached Figure Description

[0019] Figure 1 A flowchart of an anti-interference industrial visual counting method based on spatiotemporal context awareness provided by the present invention; Figure 2 This is a schematic diagram of the structure of the external detection frame and the target area in this invention; Figure 3 This is a schematic diagram illustrating the category labeling of the counting targets in this invention; Figure 4 This is a schematic diagram of the structure of an anti-interference industrial visual counting device based on spatiotemporal context awareness provided by the present invention. Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided by the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein.

[0022] It should be understood that in the various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0023] It should be understood that in this invention, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0024] It should be understood that in this invention, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0025] It should be understood that in this invention, "B corresponding to A", "B corresponding to A", "A and B correspond", or "B and A correspond" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0026] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0027] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0028] This invention provides an anti-interference industrial visual counting method based on spatiotemporal context awareness, such as... Figure 1 As shown, it includes: S1, acquire the real-time video stream of the monitored area, identify the detection target in the real-time video stream, extract the intersection area between the detection target and the preset target area, and determine the intersection ratio of the detection target based on the intersection area.

[0029] It should be noted that this invention uses visual counting of calcium carbide pots in a calcium carbide furnace workshop as an example for specific illustration. In industrial settings, such as calcium carbide furnace workshops, the environment is complex and subject to numerous interferences. If only video images are identified, irrelevant equipment in the background outside the work area is prone to false detections. Therefore, it is first necessary to limit the detection range to a specific target area. The target area refers to the pre-defined area for visual counting.

[0030] This step involves acquiring surveillance video of the monitored area using industrial cameras deployed at the industrial site and pulling the real-time video stream. Then, the latest frame of the real-time video stream is analyzed to identify the target in the video frame, specifically the opening of the calcium carbide pot, and an outer detection bounding box is generated for the target. Next, the overlap area between the outer detection bounding box and the target area is calculated, i.e., the intersection area. Finally, the area of ​​the outer detection bounding box is used as the outer area, and the ratio of the intersection area to the outer area is calculated to obtain the intersection ratio. Based on the intersection ratio, it is determined whether the detected target has effectively entered the target area.

[0031] The monitoring area refers to the area covered by the lens of an industrial camera in the industrial site where video is captured; the real-time video stream refers to the real-time video data collected by the industrial camera installed on site; the detection target refers to the object that needs to be identified and detected, and in this embodiment, it specifically refers to the mouth of the calcium carbide pot. The mouth of the pot is selected instead of the entire pot body to avoid the calcium carbide pots being blocked by each other during movement, which would lead to false detection.

[0032] In some embodiments, step S1 (identifying the detection target in the real-time video stream, extracting the intersection region between the detection target and a preset target region, and determining the intersection ratio of the detection target based on the intersection region) includes S11-S13: S11 identifies the target in the real-time video stream and generates the outer detection box of the target.

[0033] Understandably, this involves identifying the latest frame of a real-time video stream, pinpointing the target within the video frame, and generating a bounding box for that target. For example... Figure 2 As shown in the figure, the yellow rectangle represents the target area, and the green rectangle represents the circumscribed detection box.

[0034] The circumscribed detection box refers to the smallest rectangular box that encloses the target being detected.

[0035] S12, extract the intersection region between the outer detection frame and the target region, and calculate the area of ​​the intersection region as the intersection area.

[0036] The intersection region refers to the area where the outer detection frame of the detection target overlaps with the preset target area; the intersection area refers to the area of ​​the intersection region.

[0037] S13, obtain the area of ​​the outer detection box as the outer area, and obtain the intersection ratio based on the ratio of the intersection area to the outer area.

[0038] It is easy to understand that the ratio of the intersection area to the circumscribed area is used to obtain the intersection percentage.

[0039] S2, when the intersection ratio is determined to be greater than or equal to a preset intersection threshold, the corresponding detection target is used as a counting target, and video recording is performed on the counting target until the preset recording termination condition is met, and the target video segment corresponding to the counting target is obtained.

[0040] It should be noted that traditional counting methods typically employ tripwire detection, meaning that the count is immediately recorded once the target touches a pre-set virtual line. However, in actual industrial production, the target, such as a calcium carbide pot, may repeatedly move in and out of the edge. If the count is based solely on the instantaneous action of touching the virtual line, it is prone to double counting.

[0041] Therefore, this invention uses video recording. When a detection target meets the triggering condition, it does not immediately start counting, but instead records a video of the detection target that meets the condition. This allows for the acquisition of complete data on the detection target throughout the entire operation. Subsequently, the motion state of the detection target can be analyzed based on a complete video rather than a single image, thereby eliminating the count of detection targets that do not meet actual production requirements.

[0042] Understandably, the intersection ratio calculated in step S1 is continuously monitored. When the intersection ratio of a detected target is greater than or equal to a preset intersection threshold, the detected target is determined to have effectively entered the target area and is marked as a counted target. At this time, video recording for the counted target is initiated. It should be noted that several seconds of historical footage before the start of recording can be written in, for example, 5 seconds before the start of recording, so that the target video clip covers the entire process of the counted target entering the target area. Then, real-time footage is continuously recorded until the preset recording termination condition is met, resulting in the target video clip corresponding to the counted target.

[0043] Here, the intersection threshold is a pre-set value used to determine whether a detected target has effectively entered the target area; the count target is a detected target whose intersection ratio exceeds the intersection threshold; the target video segment is video data generated for the count target; and the recording termination condition is a pre-set condition used to determine whether to end the current video recording task. In this embodiment, the recording termination condition includes two conditions, and the current video recording stops when either one is met. Specifically, the recording termination conditions include: the duration of a single video recording reaches the preset recording duration, and the count target is detected to have left the target area and has not intersected with the target area again within the subsequent preset waiting time.

[0044] In some embodiments, step S2 (recording the counting target until a preset recording termination condition is met, and obtaining the target video segment corresponding to the counting target) includes S21: S21, record video of the counting target until the video recording duration reaches the preset recording duration, or when the counting target and the target area do not intersect and the duration of non-intersection reaches the preset waiting duration, stop video recording and obtain the target video segment corresponding to the counting target.

[0045] Understandably, on the one hand, when recording video of the target, the recording duration is recorded. If the recording duration reaches the preset recording duration, such as 1 hour, the recording stops. On the other hand, when recording video of the target, the intersection between the target and the target area is detected in real time. If the target and the target area do not intersect and the duration of the non-intersection reaches the preset waiting time, such as 1 minute, it is determined that the target has completely left the target area, and the recording stops.

[0046] Among them, the preset recording duration refers to the maximum allowed duration of video recording; the preset waiting duration refers to the time during which video recording continues after the counting target and the target area no longer intersect.

[0047] S3, extract the instantaneous information of the counting target in each video frame of the corresponding target video segment, and determine the detection result of the corresponding counting target based on the instantaneous information.

[0048] It should be noted that in industrial settings, factors such as flickering light, smoke obstruction, or equipment vibration can lead to inaccuracies in the recognition results based solely on a single frame image. Therefore, this invention analyzes the instantaneous information within the target video segment recorded in step S2 to obtain the detection results for the corresponding counted targets.

[0049] Among them, video frame refers to the image of each timestamp in the target video segment; instantaneous information refers to the instantaneous data of the counted target in a certain video frame of the target video segment, including the instantaneous position and instantaneous category of the counted target; detection result refers to the result generated by comparing the decision category and decision displacement direction of the counted target with the preset effective direction and effective category.

[0050] In some embodiments, step S3 (extracting instantaneous information of the counting target in each video frame of the corresponding target video segment, and determining the detection result of the corresponding counting target based on the instantaneous information) includes S31-S34: S31, the instantaneous information includes instantaneous location and instantaneous category.

[0051] In this context, "instantaneous position" refers to the position of the counted target in a certain video frame of the target video segment; "instantaneous category" refers to the category identified for the counted target in a certain video frame. For example, in this embodiment, the instantaneous category may include a full pot and an empty pot.

[0052] It should be noted that if identification is performed based on only a single video frame, the results are often uncertain. For example, a video frame may identify a pot as full, while the next video frame may identify it as empty due to reflection.

[0053] Therefore, this step reads the target video segment generated in step S2 frame by frame. For each video frame, the instantaneous position of the target is extracted, the instantaneous category of the target is identified and labeled, and the labeling scope only includes the rim of the calcium carbide pot, for example... Figure 3 The green area indicates an empty calcium carbide pot, while the red area indicates a full calcium carbide pot.

[0054] S32, generate a position sequence based on multiple instantaneous positions of the counting target, and determine the decision displacement direction of the counting target based on the position sequence.

[0055] Understandably, a position sequence is generated based on the instantaneous position of the target in each video frame corresponding to the target video segment. Subsequently, this position sequence is analyzed. Specifically, the sampling window length can be set, the local displacement vector of the target within the sampling window can be calculated, and the direction of the decision displacement of the target can be determined by statistically analyzing the direction of these local displacement vectors.

[0056] Among them, the position sequence refers to the ordered set formed by extracting the instantaneous positions of the counting target in each frame of the target video segment and arranging these instantaneous positions in chronological order; the decision displacement direction refers to the final direction of the counting target's movement determined statistically based on the entire position sequence.

[0057] In some embodiments, step S32 (determining the decision displacement direction of the counting target based on the position sequence) includes S321-S323: S321, perform sliding sampling on the position sequence based on the preset sampling window length to obtain multiple sampling sequences.

[0058] Understandably, a sampling window length, such as N, is pre-defined. Starting from the first instantaneous position in the position sequence, the first to Nth instantaneous positions are extracted as the first sampling sequence. Subsequently, the sampling window slides backward, for example with a step size of 1, extracting the second to N+1th instantaneous positions as the second sampling sequence, and so on, until the entire position sequence is traversed, ultimately resulting in multiple sampling sequences.

[0059] Here, the sampling window length refers to the sampling length that is preset for sampling, such as a sampling window length of 5 instantaneous positions; sliding sampling refers to the operation of moving the sampling window on the position sequence according to a fixed step size to continuously capture the sampling sequence; the sampling sequence refers to the set of continuous instantaneous positions obtained by sliding sampling.

[0060] S322, extract the first and last instantaneous positions of each sampling sequence, calculate the local displacement vector of each sampling sequence based on the first and last instantaneous positions, and determine the local displacement direction of each sampling sequence based on the local displacement vector.

[0061] Understandably, for each sampling sequence, the first instantaneous position is extracted as the first instantaneous position, and the last instantaneous position is extracted as the last instantaneous position. The coordinates of the first instantaneous position are subtracted from the coordinates of the last instantaneous position to obtain the local displacement vector of the corresponding sampling sequence. Next, the horizontal and vertical components of the local displacement vector are analyzed, and these two components are compared with preset spatial direction categories to determine the local displacement direction of the sampling sequence. Here, the spatial direction category refers to a pre-defined set of classifications used to characterize the direction of motion of the counting target in a two-dimensional plane; the setting of the spatial direction category can be adjusted according to the actual industrial environment.

[0062] For example, the preset spatial direction categories are: right, left, up, and down. Right corresponds to the absolute value of the horizontal component being greater than the absolute value of the vertical component, and the horizontal component being positive; left corresponds to the absolute value of the horizontal component being greater than the absolute value of the vertical component, and the horizontal component being negative; up corresponds to the absolute value of the vertical component being greater than or equal to the absolute value of the horizontal component, and the vertical component being negative; down corresponds to the absolute value of the vertical component being greater than or equal to the absolute value of the horizontal component, and the vertical component being positive. If the first instantaneous position of a sampled sequence is (100, 500) and the last instantaneous position is (300, 510), then the horizontal component of the local displacement vector is 300 - 100 = 200, and the vertical component is 510 - 500 = 10. Since 200 is greater than 10, and the horizontal component is positive, the local displacement direction of this sampled sequence is to the right.

[0063] Among them, the local displacement vector refers to the vector pointing from the first instantaneous position to the last instantaneous position; the local displacement direction refers to the direction of movement of the counting target within the time period corresponding to the sampling sequence.

[0064] S323, count the number of each local displacement direction as the local displacement quantity, and select the local displacement direction corresponding to the largest local displacement quantity as the decision displacement direction of the counting target.

[0065] Understandably, step S322 involves counting all local displacement directions corresponding to the position sequence output, and obtaining the number of each local displacement direction as the local displacement count. For example, if there are 5 counts for leftward, then the local displacement count for leftward is 5. The local displacement counts for each direction are compared, and the local displacement direction with the largest local displacement count is selected as the decision displacement direction for the counting target. For example, if the statistical results show that the local displacement count for rightward is 80, the local displacement count for leftward is 5, and the local displacement count for upward is 2, then rightward is determined as the decision displacement direction for this counting target.

[0066] The number of local displacements refers to the number of times each local displacement direction appears in all sampling sequences.

[0067] S33, count the instantaneous categories of the counting target, obtain the number of categories for each instantaneous category, and select the instantaneous category corresponding to the largest number of categories as the decision category of the counting target.

[0068] Understandably, the count involves statistically analyzing the frequency of occurrences of each instantaneous category of the target within the entire target video segment, using this as the category count. For example, if the instantaneous category "full pot" occurs 5 times, then the category count for "full pot" is 5. The counts of each instantaneous category are compared, and the instantaneous category with the highest count is selected as the decision category. For instance, based on the target video segment, if the instantaneous category "full pot" has 85 counts and the instantaneous category "empty pot" has 15 counts, then the instantaneous category "full pot" is determined as the decision category for the corresponding target.

[0069] Among them, the number of categories refers to the cumulative number of times a category appears in all video frames of the target video segment at any given moment; the decision category refers to the final classification of the target within the entire target video segment.

[0070] S34, generate the detection results of the corresponding counting targets based on the decision displacement direction and decision category.

[0071] It's important to note that not all targets appearing in the frame need to be counted. First, the direction must be correct. Production lines are typically unidirectional, such as outputting from left to right. If a target moves from right to left or remains stationary, it should not be counted, as this would result in double counting. Second, the category must match. Production management may only count specific categories of products, such as only counting the number of full batches. Therefore, a count will only be made when both the direction of movement and the category of the target meet the requirements.

[0072] In some embodiments, step S34 (generating the detection result of the corresponding counting target based on the decision displacement direction and decision category) includes S341-S343: S341, the detection results include valid results and invalid results.

[0073] It is easy to understand that test results include valid results and invalid results.

[0074] S342, obtain the preset effective direction and effective category, and when the decision displacement direction of the counting target is the same as the effective direction and the decision category is the same as the effective category, generate the effective result of the corresponding counting target.

[0075] Understandably, the process involves acquiring pre-defined valid directions and categories, comparing the decision displacement direction of the target with the valid directions, and comparing the decision category of the target with the valid categories. If the decision displacement direction of the target matches the valid direction and the decision category also matches the valid category, a valid result is generated for that target. For example, if the valid direction is set to the right and the valid category is "full pot," then if the decision displacement direction of a target is to the right and the decision category is "full pot," the detection result for that target is a valid result.

[0076] Among them, the effective direction refers to the pre-defined direction, which can be used to represent the direction of normal product flow; the effective category refers to the pre-defined category, which represents the category that needs to be counted; and the effective result refers to the result that the counting target meets the requirements.

[0077] S343, when the decision displacement direction of the counting target is different from the effective direction and / or the decision category is different from the effective category, an invalid result for the corresponding counting target is generated.

[0078] Understandably, if the decision displacement direction of a counting target is different from the valid direction, or the decision category is different from the valid category, or both the decision displacement direction and the decision category are different from the valid category, an invalid result is generated. For example, if the valid direction is set to the right and the valid category is "full pot": if the decision displacement direction of a counting target is to the left and the decision category is "full pot," then the detection result for that counting target is invalid; if the decision displacement direction of a counting target is to the right and the decision category is "empty pot," then the detection result for that counting target is invalid; if the decision displacement direction of a counting target is to the left and the decision category is "empty pot," then the detection result for that counting target is invalid.

[0079] Invalid results refer to results where the counting target does not meet the requirements.

[0080] S4, obtain the counting results within the preset detection time based on the detection results of each counting target.

[0081] In some embodiments, step S4 (obtaining the counting result within a preset detection time based on the detection result of each counting target) includes S41: S41, count the number of valid results within the preset detection time as the counting result.

[0082] It is easy to understand that within the preset detection time, the number of valid results is calculated as the final count result.

[0083] The preset detection time refers to the pre-set time period for statistical analysis.

[0084] See Figure 4This is a schematic diagram of the structure of an anti-interference industrial visual counting device based on spatiotemporal context awareness provided in an embodiment of the present invention. The device includes: The identification module is used to acquire real-time video streams of the monitored area, identify the detection targets in the real-time video streams, extract the intersection area between the detection targets and the preset target area, and determine the intersection ratio of the detection targets based on the intersection area. The judgment module is used to determine that when the intersection ratio is greater than or equal to a preset intersection threshold, the corresponding detection target is used as a counting target, and the counting target is recorded on video until the preset recording termination condition is met, so as to obtain the target video segment corresponding to the counting target. The detection module is used to extract instantaneous information of the counting target in each video frame of the corresponding target video segment, and determine the detection result of the corresponding counting target based on the instantaneous information; The counting module is used to obtain the counting results within a preset detection time based on the detection results of each counting target.

[0085] See Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. The electronic device 40 includes: a processor 41, a memory 42, and a computer program; wherein... The memory 42 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0086] The processor 41 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0087] Alternatively, the memory 42 can be either standalone or integrated with the processor 41.

[0088] When the memory 42 is a device independent of the processor 41, the device may further include: Bus 43 is used to connect the memory 42 and the processor 41.

[0089] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, is used to implement the methods provided in the various embodiments described above.

[0090] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0091] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0092] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A spatiotemporal context-aware, interference-resistant industrial visual counting method, characterized in that, include: S1, acquire the real-time video stream of the monitored area, identify the detection target in the real-time video stream, extract the intersection area between the detection target and the preset target area, and determine the intersection ratio of the detection target based on the intersection area; S2, when the intersection ratio is determined to be greater than or equal to the preset intersection threshold, the corresponding detection target is taken as the counting target, and the counting target is recorded in video until the preset recording termination condition is met, and the target video segment corresponding to the counting target is obtained. S3, extracting instantaneous information of the counting target in each video frame of the corresponding target video segment, and determining the detection result of the corresponding counting target based on the instantaneous information, including: The instantaneous information includes instantaneous location and instantaneous category; A position sequence is generated based on multiple instantaneous positions of the counting target, and the decision displacement direction of the counting target is determined based on the position sequence. The instantaneous categories of the counting target are statistically analyzed to obtain the number of categories for each instantaneous category. The instantaneous category with the largest number of categories is selected as the decision category for the counting target. Based on the decision displacement direction and decision category, the detection results of the corresponding counting targets are generated, including: The test results include valid results and invalid results; Obtain the preset effective direction and effective category; when the decision displacement direction of the counting target is the same as the effective direction and the decision category is the same as the effective category, generate the effective result of the corresponding counting target. When the decision displacement direction of the counting target is different from the effective direction and / or the decision category is different from the effective category, an invalid result is generated for the corresponding counting target. S4, obtain the counting results within the preset detection time based on the detection results of each counting target.

2. The method according to claim 1, characterized in that, The step S1, which involves identifying the target in the real-time video stream, extracting the intersection region between the target and a preset target region, and determining the intersection ratio of the target based on the intersection region, includes: Identify targets in a real-time video stream and generate bounding boxes for those targets; Extract the intersection region between the outer detection frame and the target region, and calculate the area of ​​the intersection region as the intersection area; The area of ​​the outer detection box is obtained as the outer area, and the intersection ratio is obtained based on the ratio of the intersection area to the outer area.

3. The method according to claim 1, characterized in that, The step S2 involves recording video of the counting target until a preset recording termination condition is met, resulting in a target video segment corresponding to the counting target. This includes: Video recording is performed on the counting target until the video recording duration reaches the preset recording duration, or when the counting target and the target area do not intersect and the duration of non-intersection reaches the preset waiting duration, the video recording stops, and the target video segment corresponding to the counting target is obtained.

4. The method according to claim 1, characterized in that, The step of determining the decision displacement direction of the counting target based on the position sequence includes: Multiple sampling sequences are obtained by sliding sampling of the position sequence based on a preset sampling window length; Extract the first and last instantaneous positions of each sampling sequence, calculate the local displacement vector of each sampling sequence based on the first and last instantaneous positions, and determine the local displacement direction of each sampling sequence based on the local displacement vector; The number of local displacements in each direction is counted as the local displacement count, and the local displacement direction corresponding to the largest local displacement count is selected as the decision displacement direction of the counting target.

5. The method according to claim 1, characterized in that, The S4 includes: The number of valid results within the preset detection time is used as the counting result.

6. An anti-interference industrial visual counting device based on spatiotemporal context awareness, characterized in that, include: The identification module is used to acquire real-time video streams of the monitored area, identify the detection targets in the real-time video streams, extract the intersection area between the detection targets and the preset target area, and determine the intersection ratio of the detection targets based on the intersection area. The judgment module is used to determine that when the intersection ratio is greater than or equal to a preset intersection threshold, the corresponding detection target is used as a counting target, and the counting target is recorded on video until the preset recording termination condition is met, so as to obtain the target video segment corresponding to the counting target. The detection module is used to extract instantaneous information of the counting target in each video frame of the corresponding target video segment, and to determine the detection result of the corresponding counting target based on the instantaneous information, including: The instantaneous information includes instantaneous location and instantaneous category; A position sequence is generated based on multiple instantaneous positions of the counting target, and the decision displacement direction of the counting target is determined based on the position sequence. The instantaneous categories of the counting target are statistically analyzed to obtain the number of categories for each instantaneous category. The instantaneous category with the largest number of categories is selected as the decision category for the counting target. Based on the decision displacement direction and decision category, the detection results of the corresponding counting targets are generated, including: The test results include valid results and invalid results; Obtain the preset effective direction and effective category; when the decision displacement direction of the counting target is the same as the effective direction and the decision category is the same as the effective category, generate the effective result of the corresponding counting target. When the decision displacement direction of the counting target is different from the effective direction and / or the decision category is different from the effective category, an invalid result is generated for the corresponding counting target. The counting module is used to obtain the counting results within a preset detection time based on the detection results of each counting target.

7. An electronic device, characterized in that, include: The method comprises a memory, a processor, and a computer program, wherein the computer program is stored in the memory and the processor executes the computer program to perform the method according to any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, is used to implement the method described in any one of claims 1 to 5.

Images