Battery detection method, device and storage medium

Abstract

The application discloses a battery detection method, device and storage medium, the method comprises: acquiring a current video frame collected by a battery operation station; matching each to-be-detected battery with a plurality of identified batteries based on at least one matching reference data, wherein the matching reference data comprises: a first coincidence degree between a trajectory of the to-be-detected battery and a trajectory of the identified battery, a motion parameter comparison result between the to-be-detected battery and the identified battery, and a similarity threshold value which is adjusted according to whether the to-be-detected battery is blocked; and obtaining a detection result of the operation of the to-be-detected battery as an operation detection result of the identified battery matched with the to-be-detected battery. The above scheme alleviates the problem that the to-be-detected battery cannot be matched with the identified battery due to blocking, thereby improving the accuracy of battery tracking.

Description

Technical Field

[0001] This application relates to the field of motion detection technology, and in particular to a battery detection method, device and storage medium. Background Technology

[0002] Currently, battery production still involves many manual processes, such as manual vacuuming, cleaning module adhesive, and polarity checks. Therefore, improper operation by personnel can lead to quality issues. The primary method for detecting violations of battery operation regulations is through inspectors reviewing surveillance videos or conducting on-site inspections to monitor whether workers are operating according to standards. However, inspectors cannot monitor every worker on every production line in real time, resulting in low efficiency and significant delays. Therefore, AI (Artificial Intelligence) detection methods are needed to identify whether personnel are operating according to regulations and to provide alerts for batteries with improper operation.

[0003] However, current AI detection methods still suffer from inaccurate battery tracking when identifying and detecting human operations on batteries, often failing to accurately locate batteries corresponding to improper operations. Summary of the Invention

[0004] This application provides at least one battery detection method, device, and storage medium to improve the accuracy of battery tracking.

[0005] The first aspect of this application provides a battery detection method, comprising: acquiring a current video frame collected from a battery operation station, wherein the current video frame includes a plurality of undetermined batteries; matching each undetermined battery with a plurality of identified batteries based on at least one matching reference data, wherein the identified batteries are batteries that have been assigned an identity in historical video frames, and the matching reference data includes at least one of the following: a comparison result between a similarity metric and a similarity threshold between the undetermined battery and the identified batteries, a first overlap degree between the trajectories of the undetermined battery and the identified batteries, and a motion parameter comparison result between the undetermined battery and the identified batteries, wherein the similarity threshold is adjusted accordingly based on whether the undetermined battery is occluded; acquiring the detection result of the operation of the undetermined battery as the operation detection result of the identified battery matched with the undetermined battery, wherein acquiring the detection result of the operation of the undetermined battery includes: performing pose estimation using the current video frame or the current battery image of the undetermined battery to obtain a first action detection result of the undetermined battery, wherein the current battery image of the undetermined battery is the image region of the undetermined battery in the current video frame.

[0006] The above scheme improves the accuracy of battery tracking by matching each undetermined battery with several identified batteries based on at least one of the following: the comparison result between the similarity metric and the similarity threshold between the undetermined battery and the identified battery; the first overlap degree between the trajectories of the undetermined battery and the identified battery; and the comparison result of the motion parameters between the undetermined battery and the identified battery.

[0007] The process of matching each undetermined battery with several identified batteries based on at least one matching reference data includes: designating at least one undetermined battery as a first undetermined battery, and matching each first undetermined battery with several identified batteries based on at least one of the comparison result and a first degree of overlap; and / or designating at least one undetermined battery as a second undetermined battery, and matching each second undetermined battery with several identified batteries based on the motion parameter comparison result.

[0008] The above solution matches the first undetermined battery with the identified battery based on the comparison results and / or the first degree of overlap, and / or matches the second undetermined battery with the identified battery based on the motion parameter comparison results, thus alleviating the problem that the undetermined battery cannot be matched with the identified battery due to occlusion.

[0009] The first undetermined battery includes undetermined batteries that do not meet the weak shading condition, and the second undetermined battery includes undetermined batteries that do not meet the shading condition but meet the weak shading condition; wherein, the degree of shading of the undetermined battery that meets the shading condition is higher than that of the undetermined battery that meets the weak shading condition.

[0010] The above scheme distinguishes different occlusion conditions by using matching degree characterization values ​​and performs matching based on different conditions, thus alleviating the problem that undetermined batteries cannot be matched with identified batteries due to occlusion.

[0011] The process involves matching each undetermined battery with several identified batteries based on at least one of the comparison results and the first degree of overlap. This includes: pairing each undetermined battery and each identified battery into battery pairs; for each battery pair, obtaining at least one similarity measure between the undetermined battery and the identified battery; comparing each similarity measure of the battery pair with its corresponding similarity threshold to obtain a comparison result, where the comparison result represents the magnitude relationship between the similarity measure and the corresponding similarity threshold; and determining a matching degree representation value for each undetermined battery and several identified batteries based on each similarity measure of the battery pair and the corresponding comparison result.

[0012] The above solution lowers the appearance feature similarity threshold corresponding to the appearance feature similarity and raises the motion consistency threshold corresponding to the motion consistency parameter for the unidentified battery that is obscured, thus relaxing the matching conditions and alleviating the problem that the unidentified battery cannot be matched with the identified battery due to obscuration.

[0013] The method further includes, before comparing the similarity measure with the corresponding similarity threshold, adjusting the similarity threshold corresponding to each similarity measure of the battery pair in response to the first undetermined battery in the battery pair being occluded.

[0014] The above solution alleviates the problem that the undetermined battery cannot be matched with the identified battery due to occlusion by caching the battery's motion parameters and matching the identified and undetermined batteries based on these parameters.

[0015] The similarity metric includes at least one of appearance feature similarity and motion consistency parameter; adjusting the similarity threshold corresponding to each similarity metric of the battery pair includes at least one of the following: lowering the appearance feature similarity threshold corresponding to the appearance feature similarity of the battery pair; raising the motion consistency threshold corresponding to the motion consistency parameter of the battery.

[0016] The above solution, by determining the preset workstation area where the battery is located, forms a complete link in the process of battery entry, detection, and exit, thus alleviating the problem of missed detection caused by the battery moving too fast.

[0017] The process involves determining the matching relationship between each first undetermined battery and several identified batteries based on the similarity metrics of battery pairs and the corresponding comparison results. This includes: determining the matching degree representation value of the battery pairs based on the similarity metrics of the battery pairs and the corresponding comparison results; designating battery pairs containing first undetermined batteries or containing occluded first undetermined batteries as target battery pairs; obtaining the first overlap degree between the trajectories of the first undetermined batteries and the identified batteries in the target battery pairs; adjusting the matching degree representation value of the target battery pairs based on the first overlap degree, wherein a larger first overlap degree results in a greater improvement in the matching degree represented by the matching degree representation value before and after adjustment; and determining the matching relationship between each first undetermined battery and several identified batteries based on the current matching degree representation value of the battery pairs.

[0018] The above scheme improves the accuracy of battery tracking by matching each undetermined battery with several identified batteries based on at least one of the following: the comparison result between the similarity metric and the similarity threshold between the undetermined battery and the identified battery; the first overlap degree between the trajectories of the undetermined battery and the identified battery; and the comparison result of the motion parameters between the undetermined battery and the identified battery.

[0019] The first overlap degree between the trajectory of the first undetermined battery and the identified battery in the target battery pair includes: obtaining the detection boxes corresponding to the identified batteries in the target battery pair in several historical frames, as several historical detection boxes; taking the central tendency statistical value of the second overlap degree between the target detection box of the first undetermined battery and each historical detection box as the first overlap degree of the target battery pair; and / or, adjusting the matching degree representation value of the target battery pair based on the first overlap degree of the target battery pair includes: determining the improvement degree of the matching priority of the target battery pair based on the first overlap degree of the target battery pair, wherein the larger the first overlap degree, the greater the improvement degree of the matching priority of the battery pair; determining the weight of the target battery pair based on the improvement degree of the matching priority of the target battery pair, and adjusting the matching degree representation value of the target battery pair using the weight.

[0020] The above scheme improves the accuracy of battery tracking by matching each undetermined battery with several identified batteries based on at least one of the following: the comparison result between the similarity metric and the similarity threshold between the undetermined battery and the identified battery; the first overlap degree between the trajectories of the undetermined battery and the identified battery; and the comparison result of the motion parameters between the undetermined battery and the identified battery.

[0021] The cached motion parameters are cached after the lost identified battery is confirmed to be lost, and are deleted from the cache pool after a first preset time after the loss is confirmed; the current motion parameters of the second undetermined battery are compared with the cached motion parameters of each lost identified battery to obtain the corresponding motion parameter comparison result; in response to the motion parameter comparison result between the second undetermined battery and a lost identified battery meeting the matching requirements, the second undetermined battery is determined to be matched with the lost identified battery.

[0022] The above scheme improves the accuracy of battery tracking by matching each undetermined battery with several identified batteries based on at least one of the following: the comparison result between the similarity metric and the similarity threshold between the undetermined battery and the identified battery; the first overlap degree between the trajectories of the undetermined battery and the identified battery; and the comparison result of the motion parameters between the undetermined battery and the identified battery.

[0023] The battery operation station corresponds to several station areas; before matching each undetermined battery with several identified batteries based on at least one matching reference data, the method further includes: for each battery detected in the current video frame, determining whether the battery has entered any station area based on the battery's target detection box; in response to a battery entering any station area, the battery is designated as an undetermined battery, and the matching of the undetermined battery with several identified batteries is performed.

[0024] The above scheme improves the accuracy of battery tracking by matching each undetermined battery with several identified batteries based on at least one of the following: the comparison result between the similarity metric and the similarity threshold between the undetermined battery and the identified battery; the first overlap degree between the trajectories of the undetermined battery and the identified battery; and the comparison result of the motion parameters between the undetermined battery and the identified battery.

[0025] The method further includes at least one of a start area, a detection area, and an exit area, wherein the plurality of workstation areas include a start area, a detection area, and an exit area, and the method further includes at least one of the following steps: in response to a pending battery entering the start area and there is no matching identified battery for the pending battery, assigning an identity to the pending battery as a new identified battery; in response to a pending battery entering the detection area, performing operation detection on the pending battery to obtain the detection result of the operation on the pending battery; and in response to a pending battery entering the exit area, stopping the operation detection on the battery.

[0026] The process includes several workstation areas, including at least one of a start area, a detection area, and an exit area. The step of determining whether a battery has entered the start area includes: detecting whether the target detection frame of the battery meets a first position requirement; and determining that the battery has entered the start area in response to meeting the first position requirement. The first position requirement includes at least one of the following: whether the coordinates of a preset position point within the target detection frame of the battery are within the first coordinate range corresponding to the start area; whether the distance between the preset position point coordinates and the start area is less than a distance threshold; and whether the movement direction of the preset position point points towards the start area. The step of determining whether a battery has entered the detection area includes: obtaining a third overlap between the target detection frame of the battery and the detection area; detecting whether the third overlap meets the overlap requirement; and determining that the battery has entered the detection area in response to meeting the overlap requirement. The step of determining whether a battery has entered the exit area includes: detecting whether the target detection frame of the battery meets a second position requirement; and determining that the battery has entered the exit area in response to meeting the second position requirement. The second position requirement includes: the coordinates of a preset position point within the target detection frame of the battery are within the second coordinate range corresponding to the exit area.

[0027] The above scheme accurately identifies the target area, which is crucial for subsequent targeted processing and analysis. Different areas correspond to different operating procedures and judgment rules, laying the foundation for accurate detection of personnel operating procedures.

[0028] Among them, several workstation areas include detection areas; before determining whether a battery enters any workstation area based on the target detection box of the battery, it also includes: using a perspective transformation correction algorithm to correct the current video frame, wherein the detection area in the corrected current video frame is a preset shape.

[0029] The above scheme accurately identifies the target area, which is crucial for subsequent targeted processing and analysis. Different areas correspond to different operating procedures and judgment rules, laying the foundation for accurate detection of personnel operating procedures.

[0030] Before matching each undetermined battery with several identified batteries, the method further includes: segmenting the initial detection box of the undetermined battery from the current video frame; performing reflection suppression and / or occlusion completion on the initial detection box and then extracting the contour to obtain the target detection box representing the undetermined battery.

[0031] The above solution determines whether the battery to be determined is occluded by statistically analyzing the discreteness of the pixels corresponding to the battery to be determined. When the battery to be determined is occluded, it can perform occlusion completion, which can alleviate the problem of inaccurate initial detection box of the battery to be determined leading to incorrect judgment of the preset workstation area.

[0032] The process of suppressing reflection in the initial detection frame includes: identifying reflective areas in the current video frame; performing morphological erosion on the reflective areas to obtain shrunken reflective areas; adjusting the initial detection frame using the shrunken reflective areas; and / or completing occlusion in the initial detection frame, including: detecting that the battery to be determined is occluded; using the outline of the unoccluded part in the initial detection frame of the battery to be determined to complete the outline of the occluded part in the initial detection frame, wherein the completed area in the completed detection frame is marked with a preset mark.

[0033] The detection steps for determining whether a battery to be determined is obscured include: statistically analyzing the pixel values ​​within the detection frame of the battery to be determined to obtain a discreteness statistical value; and determining that the battery to be determined is obscured if the discreteness statistical value is less than a preset statistical threshold.

[0034] The above solution determines whether the battery to be determined is occluded by statistically analyzing the discreteness of the pixels corresponding to the battery to be determined. When the battery to be determined is occluded, it can perform occlusion completion, which can alleviate the problem of inaccurate initial detection box of the battery to be determined leading to incorrect judgment of the preset workstation area.

[0035] The first motion detection result includes the joint position and motion posture of the operator to obtain the detection result of the operation of the battery to be determined. It also includes: using the current video frame or the current battery image of the battery to be determined to perform target detection to obtain the detection result of at least one target related to the battery to be determined, and using the detection result of at least one target to determine the second motion detection result of the battery to be determined, wherein the at least one target includes at least one of the battery to be determined itself and the components on the battery to be determined.

[0036] Before obtaining the detection result of the operation on the battery to be determined, the process includes: using the current video frame or the current battery image as the target image; determining the degree of blur of the target image; and performing corresponding deblurring processing on the target image.

[0037] Before performing motion detection on the battery and obtaining the motion detection results, the current video frame or the current battery image can be used as the target image. The degree of blurriness of the target image can be determined, and corresponding deblurring processing can be performed on the target image. This deblurring process alleviates the problem of inaccurate motion detection results caused by blurry images such as ghosting due to rapid movement.

[0038] The higher the degree of fuzziness, the larger the fuzz kernel used in the defuzzification process, and the more iterations are required.

[0039] The above solution, before performing motion detection on the battery and obtaining the motion detection result, can use the current video frame or the current battery image as the target image, determine the degree of blur in the target image, and perform corresponding deblurring processing on the target image. This deblurring process alleviates the problem of inaccurate motion detection results caused by blurry images such as motion blur due to rapid movement.

[0040] Wherein, at least one target includes a battery to be determined and a tag on the battery to be determined; using the detection results of at least one target, determining a second action detection result of the battery to be determined includes: using the detection results of at least one target, calculating the offset between the reference position point of the tag and the reference position point of the battery to be determined; using the offset to determine whether the tag pasting position of the battery to be determined is standard; and / or, before determining the second action detection result of the battery to be determined using the detection results of at least one target, further including: performing soft nonmaximum suppression on the detection results of at least one target.

[0041] The above method determines whether the label is correctly placed by calculating indicators such as the offset between the label and the reference position of the battery to be tested. Combined with the detection results of the operator's joint positions and posture, the standardization of the operator's operation can be judged from multiple dimensions, thereby improving the detection rate of non-standard operating behaviors.

[0042] The method of target detection using the current video frame or the current battery image of the undetermined battery includes: using the current video frame or the current battery image of the undetermined battery as the input image of the detection model; extracting image features from the input image using the detection model; performing at least one of the following processing on the image features using the detection model: increasing the feature weights of key channels in the input image, and using the increased feature weights to enhance the features belonging to key channels in the image features to obtain enhanced image features; performing local adaptive pooling on the image features to obtain pooled image features; and making predictions using the detection model based on the processed image features.

[0043] The key channel is selected based on the contribution of each feature channel of the input image, and the contribution of each feature channel of the input image is determined based on mutual information entropy; and / or, the key channel includes at least one component of the battery to be determined, and the at least one component includes at least one of an interface, a connecting wire, and a housing.

[0044] The above scheme, by strengthening the feature extraction of key channels, helps to reduce the false negative rate.

[0045] The second aspect of this application provides an electronic device including a memory and a processor coupled to each other, the processor being used to execute program instructions stored in the memory to implement the battery detection method of the first aspect described above.

[0046] A third aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the battery detection method described in the first aspect above.

[0047] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0049] Figure 1 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0050] Figure 2 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0051] Figure 3 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0052] Figure 4 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0053] Figure 5 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0054] Figure 6 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0055] Figure 7 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0056] Figure 8 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0057] Figure 9 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0058] Figure 10 This is a schematic flowchart of an embodiment of the battery testing method of this application;

[0059] Figure 11 This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0060] Figure 12 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0061] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0062] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0063] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0064] Currently, battery production still involves many manual processes, such as manual vacuuming, cleaning module adhesive, and polarity checks. Therefore, improper operation by personnel can lead to quality issues. The primary method for detecting violations is through inspectors reviewing surveillance videos or conducting on-site inspections to monitor whether workers are operating correctly. However, inspectors cannot monitor every line in real time, resulting in low efficiency and frequent delays. Therefore, AI (Artificial Intelligence) detection methods are needed to identify whether human operation is standardized and to provide alerts for batteries with improper operation.

[0065] However, current AI detection methods still suffer from inaccurate battery tracking when identifying and detecting human operations on batteries, often failing to accurately locate the battery corresponding to improper operation. For example, the inventors of this application discovered that when using some AI detection methods, such as DeepSORT, the DeepSORT algorithm first assigns a detection item ID (Identity Document), predicts the position of the detected target in the new frame using Kalman filtering, and then performs matching based on Mahalanobis distance and cosine distance. If a match is successful, the ID is inherited. However, when existing DeepSORT-based tracking technology is applied to battery scenarios, it suffers from a high rate of track breakage due to frequent human operations. Batteries are easily obstructed by hands, tooling, etc., and traditional DeepSORT relies solely on appearance and motion features for matching, which easily leads to tracking interruptions. If the battery corresponding to improper operation cannot be accurately located and further processed, it will still have an adverse impact on the battery production quality.

[0066] Please see Figure 1 This application provides a battery detection method, including:

[0067] S110: Acquire the current video frame obtained from the battery operation station, wherein the current video frame includes several undetermined batteries.

[0068] S120: Based on at least one matching reference data, each undetermined battery is matched with several identified batteries, wherein the identified batteries are batteries that have been assigned an identity in historical video frames. The matching reference data includes at least one of the following: the comparison result between the similarity metric and the similarity threshold between the undetermined battery and the identified battery, the first overlap between the trajectories of the undetermined battery and the identified battery, and the motion parameter comparison result between the undetermined battery and the identified battery. The similarity threshold is adjusted accordingly based on whether the undetermined battery is occluded.

[0069] For example, the current video frame acquired from the battery operation station may contain multiple different batteries. Target tracking can be performed on each battery, for instance, by assigning different numbers or labels to each different battery. This application does not limit the form used to distinguish different batteries. Taking assigning different labels to different batteries as an example, suppose there are 4 batteries in the current video frame, and there are also 4 batteries in a historical video frame (e.g., the previous video frame, and assuming that the "previous video frame" is the first frame in the video). The 4 batteries in the "previous video frame" can be assigned labels, for example, "1", "2", "3", and "4", meaning these batteries are identified. The 4 unidentified batteries in the current video frame are then matched with the 4 identified batteries in the previous frame; in other words, it distinguishes which battery in the current video frame is labeled "1" and which is labeled "2". If a battery in the current video frame matches the battery labeled "1" in the previous video frame—in other words, the battery in the current video frame and the battery labeled "1" in the previous video frame are the same battery—then the battery in the current video frame can also be assigned the label "1". Understandably, tracking and marking batteries helps to quickly locate the battery corresponding to improper operation during battery operation detection, and further take appropriate remedial measures.

[0070] When matching a battery to be identified with an identified battery, the matching can be performed by comparing the similarity metric (e.g., appearance feature similarity, motion consistency parameters, etc.) between the two batteries with a similarity threshold, as will be described in detail in subsequent embodiments. However, for obscured batteries, the similarity metric may change due to the obstruction, causing them to fail to match correctly with identified batteries and thus being incorrectly identified as new batteries. Therefore, adjusting the similarity threshold according to whether the battery to be identified is obscured can alleviate the problem of obscured batteries failing to match correctly.

[0071] S130: Obtain the detection result of the operation of the battery to be determined, and use it as the operation detection result of the identified battery that matches the battery to be determined.

[0072] The above scheme improves the accuracy of battery tracking by matching each undetermined battery with several identified batteries based on at least one of the following: the comparison result between the similarity metric and the similarity threshold between the undetermined battery and the identified battery; the first overlap degree between the trajectories of the undetermined battery and the identified battery; and the comparison result of the motion parameters between the undetermined battery and the identified battery.

[0073] In some embodiments, please refer to Figure 2In the above S120, each undetermined battery is matched with several identified batteries based on at least one matching reference data, including:

[0074] S220: Designate at least one undetermined battery as a first undetermined battery, and match each first undetermined battery with a plurality of identified batteries based on at least one of the comparison result and a first degree of overlap; and / or, designate at least one undetermined battery as a second undetermined battery, and match each second undetermined battery with a plurality of identified batteries based on the motion parameter comparison result.

[0075] For example, all undetermined batteries can be designated as first undetermined batteries, and the first undetermined batteries can be matched with a plurality of identified batteries based on at least one of the comparison results and a first degree of overlap. Alternatively, all undetermined batteries can be designated as second undetermined batteries, and each second undetermined battery can be matched with a plurality of identified batteries based on the motion parameter comparison results. Or, some undetermined batteries can be designated as first undetermined batteries, and another portion as second undetermined batteries, and the first undetermined batteries can be matched with a plurality of identified batteries based on at least one of the comparison results and a first degree of overlap, and each second undetermined battery can be matched with a plurality of identified batteries based on the motion parameter comparison results.

[0076] The first degree of overlap can be determined based on the intersection-over-union (IOU) of the target detection boxes between the battery to be determined and the identified batteries in the past n historical video frames, where n is a positive integer, such as 1, 2, 3, etc., and this application does not limit the specific value of n. The target detection box can be a box formed by the outline of the battery, or the smallest bounding rectangle surrounding the battery, etc., and this application does not limit the target detection box. For example, the first degree of overlap between the battery to be determined and the identified battery labeled "3" in the historical video frames can be the average of the intersection-over-union ratios between the target detection box of the battery to be determined and the historical detection boxes of the identified batteries labeled "3" in the past 3 historical video frames.

[0077] After obtaining the first degree of overlap (assuming it is 80%), the matching priority can be further increased (e.g., 30%), and then the matching degree characterization value between the undetermined battery and the identified battery can be adjusted based on the matching priority.

[0078] In some embodiments, the first undetermined battery includes undetermined batteries that do not meet the weak shading condition, and the second undetermined battery includes undetermined batteries that do not meet the shading condition but meet the weak shading condition; wherein, the degree of shading of the undetermined battery that meets the shading condition is higher than that of the undetermined battery that meets the weak shading condition.

[0079] For example, the occlusion condition or the weak occlusion condition can be determined by the variance of the pixels within the target detection box of the battery. For instance, an occlusion condition is met when the variance is less than 0.2, and a weak occlusion condition is met when the variance is between 0.3 and 0.5, etc., and this application does not impose any limitations. For example, if half of a battery is occluded and the variance of the pixels within its target detection box is 0.1, then the battery meets the occlusion condition. If one corner of a battery is occluded and the variance of the pixels within its target detection box is 0.4, then the battery meets the weak occlusion condition. In other words, the larger the variance of the pixels within the target detection box, the smaller the area of ​​the battery that is occluded; the smaller the variance of the pixels within the target detection box, the larger the area of ​​the battery that is occluded.

[0080] In some embodiments, please refer to Figure 3 In step S220 above, based on at least one of the comparison result and the first degree of overlap, each first undetermined battery is matched with a plurality of identified batteries, including:

[0081] S321: Each of the first undetermined batteries and each of the marked batteries are paired up to form battery pairs.

[0082] S322: For each battery pair, obtain at least one similarity measure between the first undetermined battery in the battery pair and the identified battery.

[0083] For example, the similarity metric may include at least one of appearance feature similarity and motion consistency parameter. Appearance feature similarity may be cosine similarity, and motion consistency parameter may be Mahalanobis distance, etc. The appearance feature cosine similarity and Mahalanobis distance between the target detection box of the undetermined battery and the identified battery in a battery pair can be obtained based on the DEEPSORT algorithm. It should be understood that the higher the appearance feature cosine similarity, the more similar the undetermined battery and the identified battery are; the smaller the Mahalanobis distance, the more similar the undetermined battery and the identified battery are.

[0084] S323: For each similarity measure of the battery pair, compare the similarity measure with the corresponding similarity threshold to obtain the comparison result of the similarity measure, wherein the comparison result represents the magnitude relationship between the similarity measure and the corresponding similarity threshold.

[0085] When the similarity metric includes cosine similarity of appearance features and Mahalanobis distance, the similarity threshold can include both the cosine similarity threshold and the Mahalanobis distance threshold. For example, the similarity threshold corresponding to cosine similarity of appearance features is the cosine similarity threshold, and the similarity threshold corresponding to Mahalanobis distance is the Mahalanobis distance threshold.

[0086] S324: Based on the similarity measures of the battery pairs and the corresponding comparison results, determine the matching relationship between each first undetermined battery and several identified batteries.

[0087] For example, in S324, the matching degree characterization value of the target battery pair can be determined based on the similarity metric and its comparison result. In some embodiments, the matching degree characterization value can be represented by a matching cost. For example, in S324, a matching cost matrix can be determined based on the similarity metric and its comparison result. The matching cost matrix includes the matching costs corresponding to several battery pairs, and then the matching battery pairs are selected based on the matching cost matrix.

[0088] In some embodiments, before comparing the similarity measure with the corresponding similarity threshold in S323, the method may further include: adjusting the similarity threshold corresponding to each similarity measure of the battery pair in response to the first undetermined battery in the battery pair being occluded.

[0089] In some embodiments, the similarity measure includes at least one of appearance feature similarity and motion consistency parameter; adjusting the similarity threshold corresponding to each similarity measure of the battery pair includes at least one of the following: reducing the appearance feature similarity threshold corresponding to the appearance feature similarity of the battery pair; increasing the motion consistency threshold corresponding to the motion consistency parameter of the battery.

[0090] In some embodiments, please refer to Figure 4 In step S324, determining the matching relationship between each first undetermined battery and several identified batteries based on the similarity measures of the battery pairs and the corresponding comparison results may include:

[0091] S4241: Based on the various similarity measures of the battery pairs and the corresponding comparison results, determine the matching degree characterization value of the battery pairs.

[0092] S4242: Select a battery pair containing a first undetermined battery or a first undetermined battery that is obscured as the target battery pair.

[0093] In other words, all undetermined batteries in the current video frame can be used as the first undetermined batteries, and the battery pair containing any of the first undetermined batteries can be used as the target battery pair. Alternatively, all undetermined batteries in the current video frame can be used as the first undetermined batteries, and the battery pair containing the occluded first undetermined battery can be used as the target battery pair.

[0094] S4243: Obtain the first degree of overlap between the trajectory of the first undetermined cell and the identified cell in the target cell pair.

[0095] S4244: Based on the first degree of overlap of the target battery pair, adjust the matching degree characterization value of the target battery pair, wherein the greater the first degree of overlap, the greater the improvement in the matching degree represented by the matching degree characterization value before and after the adjustment.

[0096] In other words, in S4244, the matching degree characterization value of the battery pair containing any of the first undetermined batteries can be adjusted according to the first overlap degree. Alternatively, the matching degree characterization value of the battery pair containing the first undetermined battery that is obscured can also be adjusted according to the first overlap degree.

[0097] For example, when the first overlap is 30%, the matching degree representation value can be improved by 30% before and after adjustment; when the first overlap is 40%, the matching degree representation value can be improved by 40%. Alternatively, when the first overlap is 30%, the matching degree representation value can be improved by 30% before and after adjustment; when the first overlap is 40%, the matching degree representation value can be improved by 50%.

[0098] S4245: Based on the current matching degree characterization value of the battery pair, determine the matching relationship between each first undetermined battery and several identified batteries.

[0099] For example, in S4245, the matching degree characterization value can be represented by the matching cost. For instance, the matching cost matrix can be determined based on the similarity metric and its comparison results. The matching cost matrix includes the matching costs corresponding to several battery pairs. The matching cost of the target battery pair is adjusted based on the first overlap of the target battery pair. Then, the matching battery pair is selected based on the adjusted matching cost matrix.

[0100] In some embodiments, obtaining the first overlap between the trajectory of the first undetermined battery and the identified battery in the target battery pair in S4243 may include: obtaining the detection boxes corresponding to the identified batteries in the target battery pair in several historical frames, as several historical detection boxes; and taking the central tendency statistics of the second overlap between the target detection box of the first undetermined battery and each historical detection box as the first overlap of the target battery pair.

[0101] For example, the outline of the identified battery can be identified and used as a historical detection frame; alternatively, the smallest bounding rectangle of the identified battery can be used as a historical detection frame. Those skilled in the art can obtain historical detection frames based on existing knowledge, and this application does not limit the form or method of obtaining historical detection frames.

[0102] The central tendency statistic can be one of the following: average, weighted average, median, etc.

[0103] For example, the second overlap degree between the target detection box of the first battery to be determined and each historical detection box can be the intersection-over-union (IOU) ratio between the target detection box of the first battery to be determined and each historical detection box. The first overlap degree can be determined based on the IOU ratio of the historical detection boxes between the battery to be determined and the identified batteries in the past n historical video frames, where n is a positive integer, such as 1, 2, 3, etc., and the specific value of n is not limited in this application. For example, the first overlap degree between the battery to be determined and the identified battery labeled "3" in the historical video frame can be the average of the IOU ratios between the target detection box of the battery to be determined and the historical detection boxes of the identified battery labeled "3" in the past 3 historical video frames.

[0104] In some embodiments, adjusting the matching degree representation value of the target battery pair based on the first overlap of the target battery pair in S4244 may include: determining the improvement degree of the matching priority of the target battery pair based on the first overlap of the target battery pair, wherein the greater the first overlap, the greater the improvement degree of the matching priority of the battery pair; determining the weight of the target battery pair based on the improvement degree of the matching priority of the target battery pair, and adjusting the matching degree representation value of the target battery pair using the weight.

[0105] For example, the matching degree representation value can be adjusted stepwise based on the first degree of overlap. For instance, when the first degree of overlap exceeds 80%, the matching priority can be 30%, and the increase in the matching degree representation value before and after adjustment can be 30%; when the first degree of overlap exceeds 60%, the matching priority can be 20%, and the increase in the matching degree representation value before and after adjustment can be 20%; when the first degree of overlap exceeds 50%, the matching priority can be 10%, and the increase in the matching degree representation value before and after adjustment can be 10%, and so on. As another example, the matching degree representation value can also be adjusted linearly based on the first degree of overlap. The specific relationship between the matching degree representation value and the first degree of overlap is not limited in this application.

[0106] After obtaining the first overlap (assuming it is 80%), the matching priority is further increased (e.g., 30%), and then the weight of the target battery pair is determined. For example, the weight is reduced by 30% from the initial weight (e.g., 100%) (i.e., the new weight is 70%), and then the matching degree representation value between the first undetermined battery and the identified battery in the target battery pair is adjusted, for example, the matching degree representation value is adjusted to 70% of the original value.

[0107] In some embodiments, please refer to Figure 5 In step S220, based on the motion parameter comparison results, each second undetermined battery is matched with several identified batteries, which may include:

[0108] S221: Obtain the cached motion parameters of each lost identified battery from the cache pool, wherein the cached motion parameters are cached after the lost identified battery is determined to be lost, and are deleted from the cache pool after a first preset time after the loss is determined.

[0109] For example, suppose a battery labeled "3" exists in a historical video frame, but is not present in the current video frame. In this case, the battery labeled "3" is the lost battery. It should be understood that the absence of the battery labeled "3" in the current video frame could be due to the battery genuinely not existing in the current frame, or it could be that the battery labeled "3" was mistakenly identified as another battery or a new battery.

[0110] For example, cached motion parameters may include velocity, direction, etc., and this application does not limit the cached motion parameters. For example, current motion parameters, such as velocity and direction, can be obtained from the Kalman filter in the DEEPSORT algorithm. This application does not limit the form or method of obtaining motion parameters.

[0111] The first preset time can be 3 seconds, 5 seconds, etc., and can be selected according to actual needs. This application does not limit the value of the first preset time. As mentioned above, if the identified battery with the number "3" is not found in the current video frame, it may be that the identified battery with the number "3" is indeed not present in the current video frame, or it may be that the identified battery with the number "3" has been misidentified as another battery or a new battery. Therefore, if the identified battery with the number "3" cannot be found in the current video frame, it can be assumed that the "lost identified battery" is a battery that has been misidentified due to obstruction or other reasons. However, if the "lost identified battery" cannot be identified after a relatively long time (i.e., the first preset time, such as 3 seconds), it may be because the battery has really disappeared. Therefore, the cached motion parameters of the lost identified battery can be deleted from the cache pool to reduce cache space usage. By deleting the cached motion parameters from the cache pool after the first preset time, the problem of expired cached motion parameters occupying too much cache space and thus reducing processing performance can be alleviated.

[0112] S222: Compare the current motion parameters of the second undetermined battery with the cached motion parameters of each lost identified battery to obtain the corresponding motion parameter comparison results.

[0113] S223: In response to the fact that the motion parameter comparison result between the second undetermined battery and a lost identified battery meets the matching requirements, determine that the second undetermined battery matches the lost identified battery.

[0114] In other words, the cached motion parameters between the pending battery and the lost identified battery can be compared. If the comparison result meets the matching requirements, it means that the second pending battery is the lost identified battery. By matching motion parameters, the problem of the second pending battery being unable to match with the identified battery due to reasons such as occlusion can be alleviated.

[0115] In some embodiments, the battery operation station corresponds to several workstation areas.

[0116] In some embodiments, please refer to Figure 6 Before matching each undetermined battery with several identified batteries based on at least one matching reference data in S120, the process may further include:

[0117] S111: For each battery detected in the current video frame, determine whether the battery has entered any workstation area based on the target detection box of the battery;

[0118] S112: In response to a battery entering any workstation area, the battery is designated as a pending battery, and the pending battery is matched with several identified batteries.

[0119] For example, any workstation area may include a "detection area", a "start area", an "exit area", etc. The coordinate range of the "detection area" in the video frame can be pre-defined, and then when the target detection box corresponding to the battery enters the coordinate range, it is regarded as a battery to be determined.

[0120] In some embodiments, the method may further include: in response to a pending battery entering a starting region and the pending battery not having a matching identified battery, assigning an identity to the pending battery as a new identified battery. In other words, if the pending battery does not match any identified battery, it indicates that the pending battery is a new battery.

[0121] For example, the starting region can be located before the detection region.

[0122] In some embodiments, the method may further include: in response to a battery to be determined entering a detection area, performing operation detection on the battery to be determined to obtain a detection result of the operation on the battery to be determined.

[0123] In some embodiments, the method may further include: stopping operation detection of the battery in response to the battery entering the departure area.

[0124] In some embodiments, the step of determining whether the battery has entered the starting region may include: detecting whether the target detection frame of the battery meets a first position requirement; and determining that the battery has entered the starting region in response to meeting the first position requirement; wherein the first position requirement includes at least one of the following: whether the coordinates of a preset position point within the target detection frame of the battery are within a first coordinate range corresponding to the starting region, whether the distance between the preset position point coordinates and the starting region is less than a distance threshold, and whether the movement direction of the preset position point points towards the starting region. Exemplarily, the preset position point can be any point within the target detection frame, and this application does not impose any limitation.

[0125] In some embodiments, the step of determining whether a battery has entered the detection area may include:

[0126] Obtain the third overlap between the target detection box of the battery and the detection area, and check whether the third overlap meets the overlap requirement. In response to meeting the overlap requirement, determine that the battery has entered the detection area.

[0127] In some embodiments, the step of determining whether a battery has entered an exit area may include:

[0128] The system detects whether the target detection box of the battery meets the second position requirement. In response to meeting the second position requirement, it determines that the battery has entered the exit area. The second position requirement includes: the coordinates of a preset position point within the target detection box of the battery are within the second coordinate range corresponding to the exit area.

[0129] For example, the position of the module in the image can be determined by calculating the center coordinates of the target detection box. The center coordinates are then compared with the coordinate ranges of a pre-defined start area, detection area, and exit area. If the center coordinates are within the start area range (i.e., the first coordinate range) and the movement direction is towards the start area, the battery is in the start area. If they are within the detection area coordinate range, the battery has entered the detection area. If they are within the exit area coordinate range, the battery has entered the exit area. Accurately determining the target's location is crucial for subsequent targeted processing and analysis. Different areas correspond to different operating procedures and judgment rules, laying the foundation for accurate detection of personnel operating procedures.

[0130] In some embodiments, before determining whether a battery has entered any workstation area based on the target detection box of the battery, the method may further include: correcting the current video frame using a perspective transformation correction algorithm, wherein the detection area within the corrected current video frame is a preset shape.

[0131] The preset shape can be a rectangle, rhombus, etc., and this application does not impose any restrictions.

[0132] For example, to improve the accuracy of coordinate calculation, perspective transformation correction technology can be introduced. Four reference points with known coordinates can be set in the actual workstation area, such as the four corners. For each reference point, its precise coordinate value in a preset physical coordinate system can be obtained using physical measurement tools (such as a laser rangefinder). The preset physical coordinate system can be set according to actual detection needs; for example, a two-dimensional Cartesian coordinate system can be established with the lower left corner reference point of the starting area as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis. An image acquisition device (such as a camera) is used to capture images of the scene containing the workstation area, obtaining the original workstation area image. In the original workstation area image, the pixel positions corresponding to the four reference points are located using image recognition algorithms (such as threshold segmentation, edge detection, template matching, etc.), or the pixel coordinates of each reference point in the image can be determined through manual marking. The physical coordinates and corresponding pixel coordinates of each reference point obtained above are converted into the input format required by the cv2.getPerspectiveTransform() function, i.e., converted into a NumPy array type. Calling the `cv2.getPerspectiveTransform(src, dst)` function outputs the perspective transformation matrix M, which contains the pixel coordinate mapping parameters from the original image to the corrected image. Calling the `cv2.warpPerspective` function transforms the coordinates of each pixel in the original image using the mapping relationship defined by the perspective transformation matrix M, resulting in the corrected image where the workstation area appears as a rectangle. At this point, the calculated coordinates have a more direct correspondence with the actual physical coordinates.

[0133] In some embodiments, please refer to Figure 7 Before matching each undetermined battery with several identified batteries in S120, the method may further include:

[0134] S711: Segment the initial detection box of the battery to be determined from the current video frame.

[0135] For example, initial detection boxes can be obtained using algorithms such as Mask R-CNN (Convolutional Neural Network) to initially segment the battery. Initial detection boxes can also be obtained using other methods, which are not limited in this application.

[0136] S712: After performing reflection suppression and / or occlusion completion on the initial detection box, contour extraction is performed to obtain the target detection box characterizing the battery to be determined.

[0137] When a battery is reflective, the reflective area may be very close to the surrounding background area. This can easily lead to the background area being mistaken for part of the battery during the initial detection bounding box recognition, resulting in a mismatch between the extracted target detection bounding box and the actual battery. This negatively impacts subsequent detection of the battery's location within the designated workstation area. For example, even if the battery has actually entered the detection area, the inconsistency between the initial detection bounding box and the actual battery might mistakenly indicate that the battery has not entered the detection area. Therefore, performing reflection suppression and / or occlusion completion on the initial detection bounding box before contour extraction to obtain the battery's target detection bounding box improves the accuracy of detecting the battery's location within the workstation area.

[0138] In some embodiments, suppressing reflections in the initial detection box may include: identifying reflective regions in the current video frame; performing morphological erosion on the reflective regions to obtain shrunken reflective regions; and adjusting the initial detection box using the shrunken reflective regions.

[0139] For example, after obtaining the initial detection box, the reflective area can be located using HSV(Hue, Saturation, Value) color space conversion to generate a reflective detection box. For instance, the portion of the V channel value greater than 180 can be identified as the reflective area. The reflective area located by the V channel brightness value may be larger than the actual reflective area. A "reflective area shrinkage" operation can be performed on the initial detection box. For example, a 3×3 erosion operator can be used to perform several (e.g., 1 time, 2 times, etc., this application is not limited) morphological erosions on the reflective area to preserve the true contour of the reflective area. After obtaining the reflective area, the initial detection box can be corrected. For example, the reflective area can be merged with the initial detection box. The corrected initial detection box can be used to extract the contour using the cv2.findContours() function to obtain the target detection box of the battery.

[0140] In some embodiments, occlusion completion of the initial detection box includes:

[0141] If the unobstructed part of the battery to be determined is detected, the outline of the obstructed part in the initial detection frame of the battery to be determined is completed using the outline of the unobstructed part in the initial detection frame. The completed area in the completed initial detection frame is marked with a preset mark.

[0142] For example, when occlusion is detected, the contour fragments of the unoccluded region (e.g., extracted using the cv2.approxPolyDP function) can be used to fit the complete contour trend. Alternatively, a "symmetric interpolation" method can be used to generate a complete detection box for the occluded region, for example, using the battery centroid as the axis of symmetry and filling in the missing portion of the occluded side with the contour features of the unoccluded side.

[0143] The completed area in the initial detection box after completion can be marked with a preset mark: the completed area can be marked with a special pixel value (such as 180) to distinguish it from the real segmented area (such as a pixel value of 255).

[0144] By suppressing reflections and / or completing occlusion in the initial detection frame of the battery, the problem of inaccurate initial detection frame leading to incorrect judgment of the preset workstation area can be alleviated.

[0145] In some embodiments, battery occlusion can be detected by: statistically analyzing the pixel values ​​within the detection frame of the battery to be determined to obtain a dispersion statistical value. If the dispersion statistical value is less than a preset statistical threshold, it is determined that the battery to be determined is occluded. By determining whether the battery to be determined is occluded based on the dispersion statistical value of the pixels corresponding to the battery, occlusion completion can be performed when the battery to be determined is occluded, thus mitigating the problem of inaccurate initial detection frames leading to incorrect judgments of preset workstation areas.

[0146] For example, the statistic for dispersion can be variance. The formula for calculating variance is as follows: Where N is the total number of pixels within the detection frame of the battery to be determined. For example, if the current detection frame is 100x50 pixels, then N = 100 × 50 = 5000. i Let be the intensity value of the i-th pixel. μ is the average intensity value of all pixels within the detection frame of the battery to be determined. The preset statistical threshold can be set according to actual needs, such as 0.2, 0.3, etc., and this application does not impose any restrictions. For example, in this embodiment, the preset statistical threshold can be 0.2, that is, when the variance is less than 0.2, it is determined that the battery is blocked.

[0147] It should be noted that, in some embodiments, the pixel values ​​within the detection box of the battery to be determined can be statistically analyzed to determine the occlusion status of the battery before obtaining the target detection box of the battery to be determined. In this case, the detection box of the battery to be determined is the initial detection box of the battery to be determined. In some embodiments, the pixel values ​​within the detection box of the battery to be determined can be statistically analyzed to determine the occlusion status of the battery after obtaining the target detection box of the battery to be determined. In this case, the detection box of the battery to be determined can be the target detection box of the battery to be determined.

[0148] In some embodiments, please refer to Figure 8 Obtaining the detection results of the operation on the battery to be determined may include at least one of the following steps:

[0149] S831: Perform pose estimation using the current video frame or the current battery image of the battery to be determined, and obtain the first action detection result of the battery to be determined. The current battery image of the battery to be determined is the image region of the battery to be determined in the current video frame. The first action detection result includes the joint position and action posture of the operator.

[0150] S832: Perform target detection using the current video frame or the current battery image of the battery to be determined, obtain the detection result of at least one target related to the battery to be determined, and determine the second action detection result of the battery to be determined using the detection result of at least one target, wherein the at least one target includes at least one of the battery to be determined itself and a component on the battery to be determined.

[0151] For example, a human pose estimation model (OpenPose) can be used for action detection. The entire video frame can be input into the human pose estimation model, or the battery image corresponding to the battery in the video frame can be input into the human pose estimation model; this application is not limited to this. The first action detection result of the battery is obtained using the human pose estimation model, such as the joint positions and action postures of an operator handling the battery (e.g., wiping away dust, removing adhesive, etc.).

[0152] For example, when performing motion detection on the battery to obtain motion detection results, pose estimation can be performed using the current video frame or the current battery image corresponding to the battery to be determined to obtain the first motion detection result of the battery to be determined. Then, target detection can be performed using the video frame or the current battery image corresponding to the battery to be determined to obtain the detection results of at least one target related to the battery to be determined. Finally, the second motion detection result of the battery to be determined can be determined using the detection results of at least one target.

[0153] For another example, when performing motion detection on the battery to be determined to obtain motion detection results, the pose can be estimated using video frames or the current battery image corresponding to the battery to be determined to obtain the first motion detection result of the battery to be determined.

[0154] For example, when performing motion detection on a battery to be determined to obtain motion detection results, target detection can be performed using video frames or the current battery image corresponding to the battery to be determined to obtain detection results of at least one target related to the battery, and the second motion detection result of the battery can be determined using the detection results of at least one target.

[0155] The inventors of this application have discovered that when performing motion detection on personnel operations in battery handling scenarios based on a human pose estimation model, the rapid speed of personnel operations, such as during battery cleaning where the operator's hands move quickly, can easily cause image motion blur, leading the human pose estimation model to misjudge the action posture, for example, misjudging "rapid grasping" as "no action." Therefore, in some embodiments, before performing motion detection on the battery and obtaining the motion detection result, the current video frame or the current battery image can be used as the target image to determine the degree of blur in the target image and perform corresponding deblurring processing. This deblurring processing alleviates the problem of inaccurate motion detection results caused by blurry images such as motion blur due to rapid movement.

[0156] In some embodiments, the higher the degree of blurring, the larger the blur kernel used in the deblurring process, and the more iterations are performed. Adaptively adjusting the blur kernel based on the degree of blurring can improve the deblurring effect.

[0157] For example, the degree of blur can be quantified by the Mean Absolute Error (MAE), because the more severe the motion blur, the greater the change in pixel position between two consecutive frames, and the higher the MAE value. For instance, when a person moves their hand quickly, the pixel difference in the hand area between adjacent frames is significant, and the MAE may exceed 25, which can be judged as severe blur. A larger blur kernel (e.g., 7×7) and more iterations (e.g., 12) can then be used to eliminate the motion blur.

[0158] For example, when calculating MAE, the target image can first be converted to grayscale. Then the MAE value of the grayscale target image can be calculated.

[0159] For example, when determining the degree of blur based on the MAE value of the target image, several MAE thresholds can be preset. For instance, the first MAE threshold can be set to 10, and the second MAE threshold to 25. Then, when the MAE is less than 10, it can be determined as slightly blurred; when the MAE is greater than or equal to 10 and less than or equal to 25, it can be determined as moderately blurred; and when the MAE is greater than 25, it can be determined as severely blurred. This application does not limit the specific rules for determining the degree of blur.

[0160] For example, when the MAE is 6, it can be determined as slightly blurred. For slightly blurred, a small blur kernel of size 3×3 can be used to reduce noise caused by overprocessing, and iterations are performed 5 times to achieve fast convergence. This eliminates slight ghosting and preserves hand texture details (such as finger joint contours) to meet OpenPose's recognition requirements for wrist and finger joints. When the MAE is 18, it can be determined as moderately blurred. A blur kernel of size 5×5 is switched, and iterations are performed 8 times to eliminate the overlapping ghosting between the hand and the module edge (such as the blurred boundary between the fingers and the module shell), making the contact point contour clear. The pixel error of the "finger gripping position" in the processed target image is less than or equal to 3px, thereby improving the accuracy of subsequent action detection. The blur kernel size and number of iterations used for different blur levels are only illustrative examples. In actual application scenarios, they can be set according to actual needs, and this application does not impose any restrictions.

[0161] In some embodiments, the method may further include: for each battery detected in the current video frame, determining whether the battery has entered the start area of ​​a preset workstation area based on the target detection box of the battery, wherein the start area is located before the detection area.

[0162] For example, the position of the module in the image can be determined by calculating the center coordinates of the target detection box. The center coordinates are then compared with the coordinate ranges of a pre-defined start area, detection area, and exit area. If the center coordinates are within the start area, the battery is in the start area. If they are within the detection area, the battery has entered the detection area. If they are within the exit area, the battery has entered the exit area. Accurately determining the target's location is crucial for subsequent targeted processing and analysis. Different areas correspond to different operating procedures and judgment rules, laying the foundation for accurate detection of personnel operating procedures.

[0163] In response to a battery entering the starting area, the battery is designated as a candidate battery, and target tracking and subsequent steps are performed on the process of each battery in the video in the preset work station area.

[0164] In some embodiments, the method may further include: in response to the battery not being detected in the current video frame, determining whether the battery has left the departure area in a preset workstation area, wherein the departure area is located after the detection area.

[0165] For example, it can be determined whether the battery is in the detection area or out of the detection area based on the center coordinates of the target detection box of the battery and the coordinate range of the preset departure area and detection area. For details, please refer to the embodiment on the determination of the start area, which will not be repeated here.

[0166] In response to the battery leaving the departure area, the battery motion detection is stopped.

[0167] In some embodiments, the confirmation conditions for the battery leaving the departure area include: the coordinates of a preset position point within the target detection box of the battery are within the second coordinate range corresponding to the departure area, and the time during which the detection area has not detected the battery reaches a second preset time.

[0168] The preset location point can be the center point of the target detection box, or an edge point, etc., and this application does not impose any restrictions.

[0169] The second preset time can be set according to actual needs, such as 3 seconds, 4 seconds, etc., and this application does not impose any restrictions.

[0170] In some embodiments, the method further includes: in response to the current video frame detecting that the battery is not in the starting region, obtaining a second overlap between the target detection box of the battery and the detection region. If the second overlap meets the overlap requirement, it is determined that the battery has entered the detection region.

[0171] Considering that the battery may rotate in real-world scenarios, the Intersection over Union (IoU) between the battery's minimum circumscribed rectangle and the preset workstation area can be calculated to improve the accuracy of measuring the battery's entry into the detection area. In other words, the target detection box can be the battery's minimum circumscribed rectangle. In this embodiment, the second overlap can be the IoU between the battery's minimum circumscribed rectangle and the preset workstation area. For example, the battery's minimum circumscribed rectangle can be obtained first, which can be achieved using the `cv2.minAreaRect()` function. Then, the intersection and union areas of the battery's minimum circumscribed rectangle and the preset workstation area's minimum circumscribed rectangle are calculated. When calculating the intersection area, the minimum circumscribed rectangle can be converted to vertex coordinates, and the intersection area can be calculated using methods for judging polygon intersections (such as the separating axis theorem), and then the area can be calculated using the polygon area calculation formula. The union area can be obtained by subtracting the intersection area from the sum of the areas of the two rectangles. Those skilled in the art can calculate the intersection and union areas based on existing knowledge, which will not be elaborated upon in this application.

[0172] The closer the value of the second overlap is to 1, the higher the degree to which the battery has entered the detection area. For example, the overlap requirement may include a second overlap greater than the cross-connection-to-union (CTU) threshold, such as a second overlap greater than 0.6. This application does not limit the specific value of the overlap requirement or the CTU threshold.

[0173] If the second overlap is less than the cross-connection ratio threshold, the battery is considered not to have fully entered the detection area. The battery's location is then reassessed, and its position is continuously monitored until it meets the conditions for entering the detection area. This mechanism ensures that subsequent action detection is only performed when the module achieves a certain coverage within the detection area. This mitigates inaccurate judgments caused by partial battery entry or false detection, thus improving the reliability of position detection results.

[0174] In some embodiments, at least one target includes the battery to be identified and a tag on the battery to be identified.

[0175] Please refer to Figure 9 The above-mentioned S832 may include S932: performing target detection using the current video frame or the current battery image of the battery to be determined, obtaining the detection result of at least one target related to the battery to be determined, and using the detection result of at least one target to calculate the offset between the reference position point of the label and the reference position point of the battery to be determined; and using the offset to determine whether the label pasting position of the battery to be determined is standard.

[0176] The reference location point can be the centroid of the tag and the battery to be determined, or it can be their respective center points, etc. This application does not limit the choice of the reference location point.

[0177] The above method determines whether the label is correctly placed by calculating indicators such as the offset between the label and the reference position of the battery to be tested. Combined with the detection results of the operator's joint positions and posture, the standardization of the operator's operation can be judged from multiple dimensions, thereby improving the detection rate of non-standard operating behaviors.

[0178] In some embodiments, the operational state of the battery to be determined can be based on motion detection results, such as the joint positions and postures of the operator, to characterize the standardization of the operator's operation. For example, in addition to indicating standard and non-standard states, the standardization of the operator's operation can also indicate an "abnormal warning" state. That is, when a potential risk is detected in the operator's operation, but no explicit standard has been violated, it can be marked as an "abnormal warning" to promptly remind relevant personnel to pay attention to operational safety and standardization. Detailed log records can also be generated, which may include images related to the operator's actions, intermediate data during the determination process (such as motion trajectory similarity values, label offsets, etc.), providing rich data resources for subsequent backtracking analysis of operator behavior and optimization of detection algorithms.

[0179] In some embodiments, if a battery is determined to have left the detection area, it indicates that all detection processes for that battery are complete, thereby stopping the detection of that battery's actions and recording information such as the time and state of the battery leaving the detection area. Resource release and reset operations can also be performed, such as closing the tracking thread associated with the battery and clearing temporarily stored detection data, to prepare for the detection of new batteries. This allows the system to efficiently and orderly process the next battery entering the detection area, improving the continuity and stability of real-time detection of personnel operating procedures.

[0180] In some embodiments, after acquiring the video from the battery operation station in S110 and before S120, the acquired video data can be split frame by frame into an image format, such as JPEG, for analysis and processing. The video frame images can also be preprocessed, for example, by performing grayscale conversion to remove color interference and highlight the image's structure and contour information; by using median filtering, Gaussian filtering, or other methods to reduce noise, eliminating salt-and-pepper noise, Gaussian noise, etc., and improving image quality; and by using histogram equalization and other techniques to enhance contrast, making targets in the image clearer and more discernible.

[0181] The preprocessed video frame images can be input into a pre-trained deep learning object detection model, such as the YOLOv5 model, to determine whether a battery exists in the video frame image. If a battery is found, further processing is performed on that video frame image. The YOLOv5 model is based on a convolutional neural network, extracting image features through multiple convolutional and pooling layers, using an anchor box mechanism to predict whether a battery exists in the image, and outputting a detection box containing the module location, category, and confidence score. When the confidence score of the detection box is higher than a set confidence threshold, it is determined that a battery exists in the video frame image, and the process proceeds to S120; otherwise, the process returns to continue acquiring video data and continuously monitoring the screen.

[0182] In some embodiments, before determining the second action detection result of the pending battery using the detection results of at least one target, soft non-maximum suppression can be applied to the detection results of at least one target. Traditional non-maximum suppression (NMS) directly deletes low-confidence detection boxes when the intersection-over-union ratio (IoU) exceeds a threshold, which may lead to missed detections, especially when there is high overlap between different detection boxes. Soft non-maximum suppression, however, retains more potential targets and improves the detection recall by reducing the confidence of highly overlapping detection boxes instead of directly deleting them. For example, a Bayesian optimization algorithm can be used to optimize the confidence threshold and IoU threshold, defining the objective function as a weighted sum of detection accuracy and recall. The algorithm automatically adjusts the parameters, conducts multiple experiments on the validation set, and finds the optimal parameter combination to balance detection accuracy and recall. In practical applications, for each detection box, its confidence can be updated according to the Soft-NMS formula based on its IoU value with the detection box with the highest confidence. The detection box is deleted only when the confidence is lower than the confidence threshold, thus obtaining a more accurate and complete module detection result.

[0183] In some embodiments, please refer to Figure 10When performing target detection using a video frame or a battery image corresponding to a battery within a video frame, the method may include step S1032: using the current video frame or the current battery image of the battery to be determined as the input image of the detection model, extracting image features from the input image using the detection model, and performing at least one of the following processing steps on the image features using the detection model: increasing the feature weights of key channels in the input image, and using the increased feature weights to enhance the features belonging to key channels in the image features to obtain enhanced image features; performing local adaptive pooling on the image features to obtain pooled image features; and using the detection model to make predictions based on the processed image features. By strengthening the feature extraction of key channels, it is beneficial to reduce the false negative rate.

[0184] For example, the detection model can be the YOLOv5 model. In practical applications, the specific detection model can be selected according to actual needs, and this application does not impose any restrictions.

[0185] Traditional YOLOv5 models use a fixed CBAM (Convolutional Block Attention Module) attention module in their backbone network. In some embodiments of this application, a dynamically weighted CBAM module can be embedded after the C3 module of the YOLOv5 model, introducing channel masks for key regions of the module through channel attention. For example, the feature channel contribution of key channels (e.g., "interfaces, connecting lines, shells") in the module samples can be statistically analyzed using mutual information entropy, and key channels can be assigned a weight of 1.2-1.5 times to enhance feature extraction of key regions, resulting in enhanced image features.

[0186] In some embodiments, the key channels are selected based on the contribution of each feature channel of the input image, and the contribution of each feature channel of the input image is determined based on mutual information entropy.

[0187] In some embodiments, the critical channel includes at least one component of the battery in question, and the at least one component includes at least one of an interface, a connecting wire, and a housing.

[0188] For example, the "global average pooling" of spatial attention in the YOLOv5 model can also be adjusted to "local adaptive pooling". The pooling window size can be dynamically adjusted according to the size of the battery (e.g., when the battery size is 20cm×45cm, the pooling window can be set to 16×16; when the battery size is 10cm×25cm, the pooling window can be set to 8×8; the choice of pooling window size is not limited in this application) to alleviate the problem of feature dilution of small targets. Multi-scale feature enhancement can also be achieved by embedding in layers 3, 5, and 7 of the backbone network, thereby strengthening the feature extraction of key areas such as module interfaces, connecting lines, and shells to reduce the small fault false detection rate. For example, the small fault false detection rate can be reduced from 30% to below 3%.

[0189] As another example, local adaptive pooling can be performed on image features while increasing the feature weights of key channels in the input image.

[0190] In some embodiments, when constructing the dataset, the appearance of batteries under different fault states can be simulated, such as damaged casings or loose connections. Data augmentation techniques (such as rotation, scaling, and adding noise) can be used to generate a large number of diverse images, expanding the scale and diversity of the training data. During model training, a transfer learning strategy can be employed, using weights pre-trained on the COCO (Common Objects in Context) dataset as initialization parameters, and then fine-tuning them on a custom battery dataset to accelerate model convergence and improve detection accuracy.

[0191] In some embodiments, when making predictions based on a detection model, a sliding window inference technique can be used. For a large input image, it can be divided into multiple sub-regions, and these sub-regions can be sequentially input into the model for identification. Then, the prediction results can be merged through non-maximum suppression to improve the fast and accurate identification of batteries.

[0192] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0193] Please see Figure 11 , Figure 11This is a schematic diagram of a framework of an embodiment of the electronic device of this application. The electronic device 80 includes a memory 81 and a processor 82 coupled to each other. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps in any of the battery detection method embodiments described above. In a specific implementation scenario, the electronic device 80 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0194] Specifically, processor 82 controls itself and memory 81 to implement the steps in any of the battery detection method embodiments described above. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.

[0195] Please see Figure 12 , Figure 12 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 90 stores program instructions 901 that can be executed by a processor. The program instructions 901 are used to implement the steps in any of the battery detection method embodiments described above.

[0196] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0197] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0198] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0199] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0200] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A battery testing method, characterized in that, The method includes: Acquire the current video frame obtained from the battery operation station, wherein the current video frame includes a number of undetermined batteries; Each of the undetermined batteries is matched with several identified batteries based on at least one matching reference data, wherein the identified batteries are batteries that have been assigned identities in historical video frames, and the matching reference data includes at least one of the following: a comparison result between a similarity metric and a similarity threshold between the undetermined battery and the identified batteries, a first overlap between the trajectories of the undetermined battery and the identified batteries, and a motion parameter comparison result between the undetermined battery and the identified batteries, wherein the similarity threshold is adjusted accordingly based on whether the undetermined battery is occluded; Obtaining the detection result of the operation on the undetermined battery as the operation detection result of the identified battery matching the undetermined battery, wherein obtaining the detection result of the operation on the undetermined battery includes: performing human pose estimation using the current video frame or the current battery image of the undetermined battery to obtain a first action detection result of the undetermined battery, wherein the current battery image of the undetermined battery is the image region of the undetermined battery in the current video frame; and determining a second action detection result of the undetermined battery based on the target detection result of the current video frame or the current battery image of the undetermined battery.

2. The method according to claim 1, characterized in that, The matching of each of the undetermined batteries with a plurality of identified batteries based on at least one matching reference data includes: At least one of the candidate batteries is designated as a first candidate battery, and each first candidate battery is matched with a plurality of identified batteries based on at least one of the comparison result and the first overlap degree; and / or, At least one of the undetermined batteries is designated as a second undetermined battery, and each second undetermined battery is matched with a plurality of identified batteries based on the motion parameter comparison results.

3. The method according to claim 2, characterized in that, The first undetermined battery includes the undetermined battery that does not meet the weak shading condition, and the second undetermined battery includes the undetermined battery that does not meet the shading condition but meets the weak shading condition. Among them, the degree of shading of the candidate battery that meets the shading condition is higher than that of the candidate battery that meets the weak shading condition.

4. The method according to claim 3, characterized in that, The matching of each of the first undetermined batteries with a plurality of identified batteries based on at least one of the comparison result and the first degree of overlap includes: Each of the first undetermined batteries and each of the labeled batteries are paired up to form a battery pair; For each battery pair, at least one similarity metric is obtained between the first undetermined battery in the battery pair and the identified battery. For each similarity metric of the battery pair, the similarity metric is compared with the corresponding similarity threshold to obtain the comparison result of the similarity metric, wherein the comparison result represents the magnitude relationship between the similarity metric and the corresponding similarity threshold; Based on the similarity metrics of the battery pairs and the corresponding comparison results, the matching relationship between each of the first undetermined batteries and the plurality of identified batteries is determined.

5. The method according to claim 4, characterized in that, Before comparing the similarity metric with the corresponding similarity threshold, the method further includes: In response to the first undetermined battery in the battery pair being occluded, the similarity thresholds corresponding to each similarity metric of the battery pair are adjusted.

6. The method according to claim 5, characterized in that, The similarity metric includes at least one of appearance feature similarity and motion consistency parameters; The adjustment of the similarity thresholds corresponding to each similarity metric of the battery pair includes at least one of the following: Lower the appearance feature similarity threshold corresponding to the appearance feature similarity of the battery pair; Increase the motion consistency threshold corresponding to the motion consistency parameter of the battery.

7. The method according to claim 4, characterized in that, The step of determining the matching relationship between each of the first undetermined batteries and the plurality of identified batteries based on the similarity metrics of the battery pairs and the corresponding comparison results includes: Based on the similarity measures of the battery pairs and the corresponding comparison results, the matching degree characterization value of the battery pairs is determined. The battery pair containing the first undetermined battery or the first undetermined battery that is obscured is designated as the target battery pair. Obtain the first degree of overlap between the trajectory of the first undetermined battery and the identified battery in the target battery pair; Based on the first overlap of the target battery pair, the matching degree characterization value of the target battery pair is adjusted, wherein the greater the first overlap, the greater the improvement in the matching degree represented by the matching degree characterization value before and after the adjustment. Based on the current matching degree characterization value of the battery, the matching relationship between each of the first undetermined batteries and the plurality of identified batteries is determined.

8. The method according to claim 7, characterized in that, The step of obtaining the first degree of overlap between the trajectory of the first undetermined battery and the identified battery in the target battery pair includes: Obtain the detection boxes corresponding to the identified batteries in the target battery pair in several historical frames, and use them as several historical detection boxes; The central tendency statistical value of the second overlap between the target detection box of the first undetermined battery and each of the historical detection boxes is used as the first overlap of the target battery pair. And / or, adjusting the matching degree characterization value of the target battery pair based on the first overlap of the target battery pair includes: Based on the first overlap of the target battery pair, the improvement degree of the matching priority of the target battery pair is determined, wherein the greater the first overlap, the greater the improvement degree of the matching priority of the battery pair. The weight of the target battery pair is determined based on the improvement of the matching priority of the target battery pair, and the matching degree characterization value of the target battery pair is adjusted using the weight.

9. The method according to claim 3, characterized in that, The matching of each second undetermined battery with several identified batteries based on the motion parameter comparison results includes: The cached motion parameters of each lost identified battery are obtained from the cache pool, wherein the cached motion parameters are cached after the lost identified battery is determined to be lost, and are deleted from the cache pool after a first preset time after the loss is determined; The current motion parameters of the second undetermined battery are compared with the cached motion parameters of each of the lost identified batteries to obtain the corresponding motion parameter comparison results; If the motion parameter comparison between the second undetermined battery and the lost identified battery meets the matching requirements, it is determined that the second undetermined battery matches the lost identified battery.

10. The method according to claim 1, characterized in that, The battery operation station corresponds to several workstation areas. Before matching each of the undetermined batteries with a plurality of identified batteries based on at least one matching reference data, the method further includes: For each battery detected in the current video frame, it is determined whether the battery has entered any of the workstation areas based on the target detection box of the battery; In response to the battery entering any of the workstation areas, the battery is designated as the pending battery, and the pending battery is matched with a plurality of identified batteries.

11. The method according to claim 10, characterized in that, The plurality of workstation areas include at least one of a start area, a detection area, and a departure area, and the method further includes at least one of the following steps: In response to the pending battery entering the starting area and the pending battery not having a matching identified battery, an identity is assigned to the pending battery to serve as a new identified battery; In response to the battery to be determined entering the detection area, an operation detection is performed on the battery to be determined to obtain the detection result of the operation on the battery to be determined; In response to the battery entering the departure area, the operation detection of the battery is stopped.

12. The method according to claim 10, characterized in that, The plurality of workstation areas include at least one of a start area, a detection area, and a departure area; The step of determining whether the battery has entered the starting region includes: The system detects whether the target detection frame of the battery meets a first position requirement. In response to meeting the first position requirement, it determines that the battery has entered the starting area. The first position requirement includes at least one of the following: whether the coordinates of a preset position point within the target detection frame of the battery are within the range of a first coordinate corresponding to the starting area; whether the distance between the preset position point coordinates and the starting area is less than a distance threshold; and whether the movement direction of the preset position point points towards the starting area. The step of determining whether the battery has entered the detection area includes: Obtain the third overlap degree between the target detection box of the battery and the detection area, and detect whether the third overlap degree meets the overlap degree requirement. In response to meeting the overlap degree requirement, determine that the battery has entered the detection area. The step of determining whether the battery has entered the exit area includes: Detect whether the target detection frame of the battery meets the second position requirement, and in response to meeting the second position requirement, determine that the battery has entered the departure area; wherein, the second position requirement includes: the coordinates of a preset position point within the target detection frame of the battery are within the second coordinate range corresponding to the departure area.

13. The method according to claim 10, characterized in that, The aforementioned workstation areas include a testing area; Before determining whether the battery has entered any of the workstation areas based on the target detection box of the battery, the method further includes: The current video frame is corrected using a perspective transformation correction algorithm, wherein the detection area within the corrected current video frame has a preset shape.

14. The method according to claim 1, characterized in that, Before matching each of the undetermined batteries with a plurality of identified batteries, the method further includes: The initial detection frame of the battery to be determined is segmented from the current video frame; After performing reflection suppression and / or occlusion completion on the initial detection box, contour extraction is performed to obtain the target detection box characterizing the battery to be determined.

15. The method according to claim 14, characterized in that, The reflection suppression is performed on the initial detection frame, including: Identify the reflective areas from the current video frame; The reflective area is morphologically etched to obtain a shrunken reflective area; The initial detection frame is adjusted using the shrunken reflective area; And / or, perform the occlusion completion on the initial detection box, including: If the unobstructed portion of the battery to be determined is detected to be blocked, the outline of the blocked portion in the initial detection frame of the battery to be determined is used to complete the outline of the unobstructed portion in the initial detection frame. The completed area in the initial detection frame is marked with a preset mark.

16. The method according to any one of claims 1 to 15, characterized in that, The detection steps for whether the battery to be determined is obstructed include: The pixel values ​​within the detection frame of the battery to be determined are statistically analyzed to obtain the discreteness statistics. In response to the dispersion statistic being less than a preset statistical threshold, it is determined that the battery to be determined is blocked.

17. The method according to claim 1, characterized in that, The first motion detection result includes the operator's joint position and motion posture. The second motion detection result for the battery to be determined, based on the target detection result of the current video frame or the current battery image of the battery to be determined, includes: Target detection is performed using the current video frame or the current battery image of the battery to be determined, and the detection result of at least one target related to the battery to be determined is obtained. The detection result of the at least one target is used to determine the second action detection result of the battery to be determined. The at least one target includes at least one of the battery to be determined itself and a component on the battery to be determined.

18. The method according to claim 17, characterized in that, Before obtaining the detection result of the operation of the battery to be determined, the method further includes: Use the current video frame or the current battery image as the target image; Determine the degree of blur in the target image and perform corresponding deblurring processing on the target image.

19. The method according to claim 18, characterized in that, The higher the degree of blurring, the larger the blur kernel used in the deblurring process, and the more iterations are required.

20. The method according to claim 17, characterized in that, The at least one target includes the battery to be identified and a tag on the battery to be identified; the step of determining a second action detection result of the battery to be identified using the detection result of the at least one target includes: Using the detection results of the at least one target, calculate the offset between the reference position point of the tag and the reference position point of the battery to be determined; The offset is used to determine whether the label placement of the battery to be identified is standardized. And / or, before determining the second action detection result of the battery to be determined using the detection result of the at least one target, the method further includes: Soft nonmaximum suppression is applied to the detection results of the at least one target.

21. The method according to claim 17, characterized in that, The step of performing target detection using the current video frame or the current battery image of the battery to be determined includes: Use the current video frame or the current battery image of the battery to be determined as the input image of the detection model; The detection model is used to extract image features from the input image; The image features are processed using the detection model using at least one of the following methods: Increase the feature weights of key channels in the input image, and use the increased feature weights to enhance the features belonging to key channels in the image features to obtain enhanced image features; Local adaptive pooling is performed on the image features to obtain the pooled image features; The detection model is used to make predictions based on the processed image features.

22. The method according to claim 21, characterized in that, The key channels are selected based on the contribution of each feature channel of the input image, and the contribution of each feature channel of the input image is determined based on mutual information entropy. And / or, the key channel includes at least one component of the battery in question, the at least one component including at least one of an interface, a connecting wire, and a housing.

23. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory being used to store program instructions, and the processor being used to execute the program instructions to implement the method as described in any one of claims 1-22.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the method as claimed in any one of claims 1-22.

Images