Horizontal fixing detection method and system for energy storage product

Abstract

The invention discloses a method and a system for detecting horizontal fixation of an energy storage product. The method comprises the following steps of: acquiring a first current field image and a second current field image of the energy storage product and a leveling instrument on an energy storage product horizontal fixing device through an image acquisition device respectively; the server identifies the first current field image based on an image identification model to obtain an energy storage product identification result and a first horizontal placement detection result, and detects the second current field image to obtain a second horizontal placement detection result; and when it is determined that the energy storage product information is matched and the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results, generating horizontal placement detection data of the energy storage product and sending the horizontal placement detection data to the user terminal. According to the embodiment of the invention, remote unmanned detection can be performed on the horizontal placement state of the energy storage product based on the image acquired by the image acquisition device on the energy storage product on the energy storage product horizontal fixing device, so that the efficiency of acquiring the horizontal placement state of the energy storage product is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of new energy storage product detection, and in particular to a method and system for detecting horizontal fixation of energy storage products. Background Art

[0002] With the continuous development of new energy technologies, more and more energy storage products have been applied, such as battery energy storage products. In order to fix the energy storage products firmly and horizontally at the installation location, it is necessary to use external installation auxiliary tools such as a level to complete the horizontal installation of the fixing frame (such as a fixed base or a wall-mounted device). During the installation of this fixing frame, the manual installation method using a level not only increases the complexity of the installation, but also requires manual observation of the state of the level during the installation process, resulting in a longer installation time.

[0003] If some important energy storage products need to be monitored for a long time to see if they are in a horizontal state after being fixed and installed, this is generally done manually by regularly visiting the site to check, which leads to inefficiency in obtaining the horizontal placement state of the energy storage products on the fixed frame. Summary of the invention

[0004] The embodiments of the present invention provide a method and system for detecting the horizontal fixation of an energy storage product, aiming to solve the problem in the prior art that after the energy storage product is installed on a fixing frame and after the installation is completed, it is necessary to manually check on site whether it is in a horizontal state, resulting in low efficiency in obtaining the horizontal placement state of the energy storage product on the fixing frame.

[0005] In a first aspect, an embodiment of the present invention provides a method for detecting the horizontal fixation of an energy storage product, which is applied to a system for detecting the horizontal fixation of an energy storage product, wherein the system for detecting the horizontal fixation of an energy storage product comprises a server, an image acquisition device, a device for fixing the horizontal position of an energy storage product, and a user terminal, wherein the image acquisition device and the user terminal are both communicatively connected to the server, and a level ruler is provided on the device for fixing the horizontal position of an energy storage product; the method for detecting the horizontal fixation of an energy storage product comprises: The user terminal generates a field image acquisition instruction in response to the energy storage product level detection instruction and sends the instruction to the server, so that the server sends the field image acquisition instruction to the image acquisition device; The image acquisition device acquires images of the energy storage product horizontal fixing device and the energy storage product fixed on the energy storage product horizontal fixing device based on the on-site image acquisition instruction to obtain a first current on-site image, and sends the first current on-site image to the server; The image acquisition device acquires an image of the level ruler on the horizontal fixing device of the energy storage product to obtain a second current scene image, and sends the second current scene image to the server; The server performs image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and performs target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result; If the server determines that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, and determines that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results, it generates energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction, and sends the energy storage product horizontal placement detection data to the user terminal.

[0006] In a second aspect, an embodiment of the present invention further provides a system for detecting a horizontal fixation of an energy storage product, wherein the system comprises a server, an image acquisition device, a horizontal fixation device for an energy storage product, and a user terminal, wherein the image acquisition device and the user terminal are both communicatively connected to the server, and a level ruler is provided on the horizontal fixation device for an energy storage product; The user terminal is used to generate a field image acquisition instruction in response to the energy storage product level detection instruction and send it to the server, so that the server sends the field image acquisition instruction to the image acquisition device; The image acquisition device is used to acquire images of the energy storage product horizontal fixing device and the energy storage product fixed on the energy storage product horizontal fixing device based on the on-site image acquisition instruction to obtain a first current on-site image, and send the first current on-site image to the server; The image acquisition device is further used to acquire an image of the level ruler on the horizontal fixing device of the energy storage product to obtain a second current scene image, and send the second current scene image to the server; The server is configured to perform image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and perform target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result; The server is further configured to generate energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction, and send the energy storage product horizontal placement detection data to the user terminal if it is determined that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, and if it is determined that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results.

[0007] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0008] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect above can be implemented.

[0009] The embodiment of the present invention provides a method and system for detecting the horizontal fixation of an energy storage product, wherein the system for detecting the horizontal fixation of an energy storage product comprises a server, an image acquisition device, a horizontal fixation device for an energy storage product and a user terminal, wherein the image acquisition device and the user terminal are both communicatively connected to the server, and a level ruler is provided on the horizontal fixation device for the energy storage product; the method comprises: the user terminal generates a field image acquisition instruction in response to a horizontal detection instruction for an energy storage product and sends the instruction to the server, so that the server sends the field image acquisition instruction to the image acquisition device; the image acquisition device acquires images of the horizontal fixation device for the energy storage product and the energy storage product fixed on the horizontal fixation device for the energy storage product based on the field image acquisition instruction, obtains a first current field image, and sends the first current field image to the server; the image acquisition The device performs image acquisition on the level ruler on the horizontal fixing device of the energy storage product to obtain a second current scene image, and sends the second current scene image to the server; the server performs image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and performs target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result; if the server determines that the energy storage product recognition result matches the energy storage product information uploaded by the user terminal, and determines that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results, then energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction is generated, and the energy storage product horizontal placement detection data is sent to the user terminal. The embodiment of the present invention can perform remote unmanned detection of the horizontal placement state of the energy storage product based on the image acquired by the image acquisition device on the energy storage product on the horizontal fixing device of the energy storage product, thereby improving the efficiency of obtaining the horizontal placement state of the energy storage product. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0011] Figure 1 A schematic diagram of an application scenario of the method for detecting horizontal fixation of energy storage products provided by an embodiment of the present invention; Figure 2 A schematic diagram of a flow chart of a method for detecting horizontal fixation of an energy storage product provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of a device for fixing a storage product horizontally in a method for detecting fixing a storage product horizontally provided in an embodiment of the present invention; Figure 4 A schematic diagram of a sub-process of a method for detecting horizontal fixation of an energy storage product provided by an embodiment of the present invention; Figure 5 A schematic diagram of a gesture recognition device and an interaction area in a method for detecting horizontal fixation of an energy storage product provided by an embodiment of the present invention; Figure 6 A schematic block diagram of a horizontal fixation detection system for energy storage products provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0014] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0015] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0016] Please also refer to Figure 1 and Figure 2 ,in Figure 1 Schematic diagram of a scene of a method for detecting horizontal fixation of an energy storage product according to an embodiment of the present invention. Figure 2FIG. 1 is a flow chart of a method for detecting horizontal fixation of an energy storage product provided by an embodiment of the present invention. Figure 1 As shown, the energy storage product horizontal fixation detection method provided in the embodiment of the present invention is applied to the energy storage product horizontal fixation detection system, the energy storage product horizontal fixation detection system comprises a server 10, an image acquisition device 20, an energy storage product horizontal fixation device 30 and a user terminal 40, the image acquisition device 20 and the user terminal 40 are both connected to the server 10 for communication, and the energy storage product horizontal fixation device 30 is provided with a level ruler 31 (for details, refer to Figure 3 ).

[0017] like Figure 2 As shown, the method includes the following steps S110-S150.

[0018] S110. The user terminal generates a field image acquisition instruction in response to the energy storage product level detection instruction and sends the instruction to the server, so that the server sends the field image acquisition instruction to the image acquisition device.

[0019] In this embodiment, when a user needs to obtain the horizontal placement status of a specified energy storage product on-site or remotely, the user terminal (such as a smart phone, tablet computer, etc.) can be operated to log in to the energy storage product horizontal fixed detection platform of the server, and then the energy storage product to be detected is selected from the energy storage product list on the user interaction interface corresponding to the energy storage product horizontal fixed detection platform, and this is used as the object to be detected and a corresponding energy storage product horizontal detection instruction is generated. The generated on-site image acquisition instruction is sent by the server to the user terminal. After receiving the energy storage product horizontal detection instruction and adding the user terminal device unique identifier therein, the user terminal can generate a corresponding on-site image acquisition instruction and send it to the server, and the server sends the on-site image acquisition instruction to the image acquisition terminal corresponding to the object to be detected (that is, the image acquisition terminal deployed in the same venue as the object to be detected and having a binding relationship), so as to trigger the detection of the horizontal placement status of the energy storage product as the object to be detected.

[0020] In the scenario where the user needs to obtain the horizontal placement status of a specified energy storage product on site, the user initially fixes the energy storage product on the energy storage product horizontal fixing device, and then manually checks the level ruler 31 on the energy storage product horizontal fixing device 30 (for details, please refer to Figure 3 After the situation of the level bubble in the structure of the horizontal fixing device 30 and the spirit level 31 of the energy storage product confirms that the energy storage product is in a horizontal state, the on-site image acquisition device can also collect images of the energy storage product and the spirit level respectively, and then the background server can perform another horizontal placement state detection in the dimension of image recognition.

[0021] Of course, in a scenario where the user needs to remotely obtain the horizontal placement status of a specified energy storage product, since the user is not at the site where the energy storage product is placed, the user can only indirectly capture images of the energy storage product's horizontal fixing device, energy storage product and level ruler through the image acquisition device on site and upload them to the server for image recognition.

[0022] S120. The image acquisition device acquires images of the energy storage product horizontal fixing device and the energy storage product fixed on the energy storage product horizontal fixing device based on the on-site image acquisition instruction to obtain a first current on-site image, and sends the first current on-site image to the server.

[0023] In this embodiment, when the image acquisition device corresponding to the object to be detected may specifically include multiple cameras, one of which is aimed at the horizontal fixing device of the energy storage product as the object to be detected and the energy storage product fixed on the horizontal fixing device of the energy storage product to perform image acquisition, and the other is aimed at the level ruler on the horizontal fixing device of the energy storage product to perform image acquisition. If the camera aimed at the horizontal fixing device of the energy storage product as the object to be detected and the energy storage product fixed on the horizontal fixing device of the energy storage product is recorded as the first camera, the first current scene image can be acquired through the first camera, and the first current scene image is sent to the server, and the server performs image recognition processing. Among them, when the first current scene image is subjected to image recognition by the server, the product type of the energy storage product and whether the energy storage product is placed horizontally are mainly identified.

[0024] S130: The image acquisition device acquires an image of the level ruler on the horizontal fixing device of the energy storage product to obtain a second current scene image, and sends the second current scene image to the server.

[0025] In this embodiment, if the camera aimed at the level ruler on the horizontal fixing device of the energy storage product to be detected is recorded as the second camera, the second current scene image can be collected by the second camera, and the second current scene image is sent to the server, and the server performs image recognition processing. Wherein, when the second current scene image is image recognized by the server, the main detection state corresponding to the level ruler is whether it is a horizontal state.

[0026] S140. The server performs image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and performs target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result.

[0027] In this embodiment, after the server receives the first current scene image uploaded by the image acquisition device and the second current scene image, it can perform image recognition on the first current scene image based on an image recognition model (such as a convolutional neural network, a recurrent neural network, a multi-target detection model, etc.) to obtain an energy storage product recognition result and a first horizontal placement detection result, and perform target detection on the second current scene image to obtain a second horizontal placement detection result. Based on the energy storage product recognition result, the first horizontal placement detection result, and the second horizontal placement detection result, it can be determined whether the energy storage product is still placed on the energy storage product horizontal fixing device for detection, and it can also be determined whether the energy storage product is still in a horizontal placement state.

[0028] In one embodiment, if Figure 4 As shown, in step S140, image recognition is performed on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, including: S141, performing energy storage product distribution area detection on the first current scene image based on a multi-target detection model corresponding to the image recognition model to obtain a distribution target area of ​​energy storage products; S142, identifying the product image corresponding to the distribution target area based on the multi-target detection model to obtain the energy storage product identification result; S143. Detect a boundary contour corresponding to the distributed target area based on the multi-target detection model to obtain the first horizontal placement detection result.

[0029] In this embodiment, when the first current scene image is recognized based on the image recognition model, the energy storage product distribution area of ​​the energy storage product in the first current scene image is first determined based on the multi-target detection model (such as the YOLO V5 multi-target detection model, etc.) in the image recognition model, and specifically the outer contour boundary of the energy storage product is detected to obtain the distribution target area of ​​the energy storage product. After that, the image recognition is continued for the image area corresponding to the energy storage product distribution target area in the first current scene image, and the product image corresponding to the distribution target area (i.e., the energy storage product image) is recognized by the multi-target detection model to obtain the energy storage product recognition result, and the energy storage product recognition result includes at least information such as the product name and product model. Since the target distribution area of ​​the energy storage product has been determined before, the corresponding multiple vertex coordinates in the outer contour boundary can be determined at this time. After the multiple vertex coordinates of the distribution target area, i.e., the boundary contour, are determined, the determined product distribution area characteristics can be compared with the boundary contour corresponding to the initial product placement state image collected after the energy storage product is initially installed and placed horizontally and stored locally on the server. When the two are consistent, it indicates that the first horizontal placement detection result is a horizontal placement result, and when the two are inconsistent, it indicates that the first horizontal placement detection result is a non-horizontal placement result. It can be seen that through the above method, the first current scene image can be quickly identified based on the background server to determine the energy storage product identification result and the first horizontal placement detection result.

[0030] In one embodiment, if Figure 4 As shown, in step S140, target detection is performed on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result, including: S144, identifying the positions of multiple vials in the second current scene image based on the multi-target detection model in the image recognition model to obtain multiple vial position recognition results; S145. If it is determined that the plurality of level bubble position identification results all correspond to the level bubble being located at the center of the level bubble container, the current horizontal state of the energy storage product is taken as the second horizontal placement detection result.

[0031] In this embodiment, when identifying the positions of multiple vials in the second current scene image based on the multi-target detection model in the image recognition model, for example, Figure 3Taking the example of a level ruler 31 provided with three level bubble containers (and each level bubble container contains a level bubble), if the second camera in the above example captures the second current scene image corresponding to each level bubble container in the level ruler 31 from a side view angle, the second current scene image can be regarded as a front view angle image of the three level bubble containers. At this time, the positions of multiple level bubbles in the second current scene image can be identified based on the multi-target detection model in the image recognition model (refer to the above process of identifying the outer boundary contour of the energy storage product) to obtain multiple level bubble position identification results. Then, based on whether the multiple level bubble position identification results all correspond to the situation where the level bubble is located at the center of the level bubble container, the second horizontal placement detection result is finally determined.

[0032] In one embodiment, if Figure 5 As shown, step S144 includes: S1441, detecting multiple vial bubble container areas in the second current scene image based on the multi-target detection model in the image recognition model to obtain multiple vial bubble container area detection results; S1442. If it is determined that a vial exists in the central area of ​​each of the multiple vial container area detection results, the vial located at the center of the vial container is used as the position identification result of each vial in the multiple vial position identification results.

[0033] In this embodiment, for example, after identifying the four vertices of the outer boundary contour of the level bubble container 1 among the three level bubble containers in the second current scene image, the container center point coordinate 1 of the level bubble container 1 can be determined; thereafter, after determining the four vertices of the outer boundary contour of the level bubble 1 in the level bubble container 1, the level bubble center point coordinate 1 can be determined; if it is determined that the container center point coordinate 1 coincides with the level bubble center point coordinate 1, it means that the level bubble 1 is located at the center position of the level bubble container 1. The judgment process of whether the level bubbles in other multiple level bubble containers are at the center position of the level bubble container is also referred to the above process. After obtaining the judgment results of whether the level bubbles in multiple level bubble containers are at the center position of the level bubble container, it can be comprehensively used to determine whether the energy storage product is currently in a horizontal state. Specifically, if it is determined that the multiple level bubble position recognition results all correspond to the level bubble being located at the center position of the level bubble container, the current horizontal state of the energy storage product is taken as the second horizontal placement detection result.

[0034] In one embodiment, after step S144, the method further includes: If it is determined that among the multiple vial position identification results, there is at least one vial that is not located at the center of the vial container, the current non-horizontal state of the energy storage product is taken as the second horizontal placement detection result.

[0035] In this embodiment, if it is determined that among the multiple level bubble position identification results, there is at least one level bubble that is not located at the center of the level bubble container, it means that the horizontal fixing device of the energy storage product is in a non-horizontal state, and causes the energy storage product fixed thereon to be in a non-horizontal state. At this time, the current non-horizontal state of the energy storage product is used as the second horizontal placement detection result.

[0036] S150. If the server determines that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, and determines that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results, then the server generates energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction, and sends the energy storage product horizontal placement detection data to the user terminal.

[0037] In this embodiment, if it is determined in the server that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, it means that the energy storage product initially placed on the horizontal fixing device of the energy storage product has not been removed or illegally disassembled. Moreover, at this time, it is also determined that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results, which means that the energy storage product placed on the horizontal fixing device of the energy storage product is currently still in a horizontal placement state. At this time, energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction is generated, and the energy storage product horizontal placement detection data is sent to the user terminal. After receiving the energy storage product horizontal placement detection data, the user terminal can intuitively view the horizontal placement state of the energy storage product.

[0038] In one embodiment, step S150 includes: The product name corresponding to the energy storage product fixed on the energy storage product horizontal fixing device, the product deployment position, the horizontal detection time corresponding to the energy storage product horizontal detection instruction, the first horizontal placement detection result and the second horizontal placement detection result are obtained, and filled into a preset detection data template to generate the energy storage product horizontal placement detection data.

[0039] In this embodiment, the server determines the specific product information (such as product name, product model, etc.) of the energy storage product placed on the energy storage product horizontal fixing device based on the energy storage product identification result, the first horizontal placement detection result, and the second horizontal placement detection result, and after determining that it is in a horizontal placement state, it can also obtain more specifically the product deployment position and the horizontal detection time corresponding to the energy storage product horizontal detection instruction, and finally fill the above information into the preset detection data template to generate the energy storage product horizontal placement detection data. By viewing the energy storage product horizontal placement detection data, the user terminal can obtain information such as the horizontal placement state of the energy storage product.

[0040] In one embodiment, after step S140, the method further includes: If the server determines that the energy storage product identification result does not match the energy storage product information uploaded by the user terminal, or determines that the first horizontal placement detection result is a non-horizontal placement result, or determines that the second horizontal placement detection result is a non-horizontal placement result, it generates energy storage product abnormal prompt data corresponding to the energy storage product horizontal detection instruction, and sends the energy storage product abnormal prompt data to the user terminal.

[0041] In this embodiment, if it is determined that the energy storage product identification result does not match the energy storage product information uploaded by the user terminal, it means that the energy storage product on the horizontal fixing device of the energy storage product has been moved or the equipment has been replaced; or if it is determined that the first horizontal placement detection result is a non-horizontal placement result, or if it is determined that the second horizontal placement detection result is a non-horizontal placement result, it means that the energy storage product on the horizontal fixing device of the energy storage product is not in a horizontal placement state. After any one of the above judgment conditions is met, it can be regarded that an abnormal situation has occurred in the energy storage product. At this time, the energy storage product abnormal prompt data corresponding to the energy storage product horizontal detection instruction is directly generated, and the energy storage product abnormal prompt data is sent to the user terminal. For example, the energy storage product abnormal prompt data includes the product name, product deployment location, the horizontal detection time corresponding to the energy storage product horizontal detection instruction, the first horizontal placement detection result and the second horizontal placement detection result. The above data is filled into the preset abnormal data prompt template to generate the energy storage product abnormal prompt data. The user terminal can obtain information such as the horizontal placement state of the energy storage product by viewing the energy storage product abnormal prompt data, and then determine whether to go to the site in time to handle the abnormal situation related to the energy storage product.

[0042] It can be seen that the embodiment of the method can perform remote unmanned detection of the horizontal placement state of the energy storage product based on the image captured by the image acquisition device of the energy storage product on the horizontal fixing device of the energy storage product, thereby improving the efficiency of obtaining the horizontal placement state of the energy storage product.

[0043] Figure 6 is a schematic block diagram of a horizontal fixing detection system for energy storage products provided by an embodiment of the present invention. Figure 6 As shown, corresponding to the above energy storage product horizontal fixation detection method, the present invention also provides an energy storage product horizontal fixation detection system 100. Figure 1 and Figure 6 As shown, the energy storage product horizontal fixing detection system 100 includes a server 10, an image acquisition device 20, an energy storage product horizontal fixing device 30 and a user terminal 40, the image acquisition device 20 and the user terminal 40 are both connected to the server 10 for communication, and the energy storage product horizontal fixing device 30 is provided with a level ruler 31 (for details, refer to Figure 3 ).

[0044] The user terminal 40 is used to generate a field image acquisition instruction in response to the energy storage product level detection instruction and send it to the server, so that the server sends the field image acquisition instruction to the image acquisition device.

[0045] In this embodiment, when a user needs to obtain the horizontal placement status of a specified energy storage product on-site or remotely, the user terminal (such as a smart phone, tablet computer, etc.) can be operated to log in to the energy storage product horizontal fixed detection platform of the server, and then the energy storage product to be detected is selected from the energy storage product list on the user interaction interface corresponding to the energy storage product horizontal fixed detection platform, and this is used as the object to be detected and a corresponding energy storage product horizontal detection instruction is generated. The generated on-site image acquisition instruction is sent by the server to the user terminal. After receiving the energy storage product horizontal detection instruction and adding the user terminal device unique identifier therein, the user terminal can generate a corresponding on-site image acquisition instruction and send it to the server, and the server sends the on-site image acquisition instruction to the image acquisition terminal corresponding to the object to be detected (that is, the image acquisition terminal deployed in the same venue as the object to be detected and having a binding relationship), so as to trigger the detection of the horizontal placement status of the energy storage product as the object to be detected.

[0046] In the scenario where the user needs to obtain the horizontal placement status of a specified energy storage product on site, the user initially fixes the energy storage product on the energy storage product horizontal fixing device, and then manually checks the level ruler 31 on the energy storage product horizontal fixing device 30 (for details, please refer to Figure 3 After the situation of the level bubble in the structure of the horizontal fixing device 30 and the spirit level 31 of the energy storage product confirms that the energy storage product is in a horizontal state, the on-site image acquisition device can also collect images of the energy storage product and the spirit level respectively, and then the background server can perform another horizontal placement state detection in the dimension of image recognition.

[0047] Of course, in a scenario where the user needs to remotely obtain the horizontal placement status of a specified energy storage product, since the user is not at the site where the energy storage product is placed, the user can only indirectly capture images of the energy storage product's horizontal fixing device, energy storage product and level ruler through the image acquisition device on site and upload them to the server for image recognition.

[0048] The image acquisition device 20 acquires images of the energy storage product horizontal fixing device and the energy storage product fixed on the energy storage product horizontal fixing device based on the on-site image acquisition instruction, obtains a first current on-site image, and sends the first current on-site image to the server.

[0049] In this embodiment, when the image acquisition device corresponding to the object to be detected may specifically include multiple cameras, one of which is aimed at the horizontal fixing device of the energy storage product as the object to be detected and the energy storage product fixed on the horizontal fixing device of the energy storage product to perform image acquisition, and the other is aimed at the level ruler on the horizontal fixing device of the energy storage product to perform image acquisition. If the camera aimed at the horizontal fixing device of the energy storage product as the object to be detected and the energy storage product fixed on the horizontal fixing device of the energy storage product is recorded as the first camera, the first current scene image can be acquired through the first camera, and the first current scene image is sent to the server, and the server performs image recognition processing. Among them, when the first current scene image is subjected to image recognition by the server, the product type of the energy storage product and whether the energy storage product is placed horizontally are mainly identified.

[0050] The image acquisition device 20 is further used to acquire an image of the level ruler on the horizontal fixing device of the energy storage product to obtain a second current scene image, and send the second current scene image to the server.

[0051] In this embodiment, if the camera aimed at the level ruler on the horizontal fixing device of the energy storage product to be detected is recorded as the second camera, the second current scene image can be collected by the second camera, and the second current scene image is sent to the server, and the server performs image recognition processing. Wherein, when the second current scene image is image recognized by the server, the main detection state corresponding to the level ruler is whether it is a horizontal state.

[0052] The server 10 is used to perform image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and to perform target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result.

[0053] In this embodiment, after the server receives the first current scene image uploaded by the image acquisition device and the second current scene image, it can perform image recognition on the first current scene image based on an image recognition model (such as a convolutional neural network, a recurrent neural network, a multi-target detection model, etc.) to obtain an energy storage product recognition result and a first horizontal placement detection result, and perform target detection on the second current scene image to obtain a second horizontal placement detection result. Based on the energy storage product recognition result, the first horizontal placement detection result, and the second horizontal placement detection result, it can be determined whether the energy storage product is still placed on the energy storage product horizontal fixing device for detection, and it can also be determined whether the energy storage product is still in a horizontal placement state.

[0054] In one embodiment, the server 10 is specifically used for: Performing energy storage product distribution area detection on the first current scene image based on a multi-target detection model corresponding to the image recognition model to obtain a distribution target area of ​​energy storage products; Identify the product image corresponding to the distribution target area based on the multi-target detection model to obtain the energy storage product identification result; The boundary contour corresponding to the distributed target area is detected based on the multi-target detection model to obtain the first horizontal placement detection result.

[0055] In this embodiment, when the first current scene image is recognized based on the image recognition model, the energy storage product distribution area of ​​the energy storage product in the first current scene image is first determined based on the multi-target detection model (such as the YOLO V5 multi-target detection model, etc.) in the image recognition model, and specifically the outer contour boundary of the energy storage product is detected to obtain the distribution target area of ​​the energy storage product. After that, the image recognition is continued for the image area corresponding to the energy storage product distribution target area in the first current scene image, and the product image corresponding to the distribution target area (i.e., the energy storage product image) is recognized by the multi-target detection model to obtain the energy storage product recognition result, and the energy storage product recognition result includes at least information such as the product name and product model. Since the target distribution area of ​​the energy storage product has been determined before, the corresponding multiple vertex coordinates in the outer contour boundary can be determined at this time. After the multiple vertex coordinates of the distribution target area, i.e., the boundary contour, are determined, the determined product distribution area characteristics can be compared with the boundary contour corresponding to the initial product placement state image collected after the energy storage product is initially installed and placed horizontally and stored locally on the server. When the two are consistent, it indicates that the first horizontal placement detection result is a horizontal placement result, and when the two are inconsistent, it indicates that the first horizontal placement detection result is a non-horizontal placement result. It can be seen that through the above method, the first current scene image can be quickly identified based on the background server to determine the energy storage product identification result and the first horizontal placement detection result.

[0056] In one embodiment, the server 10 is further specifically configured to: Identifying multiple vial positions in the second current scene image based on a multi-target detection model in the image recognition model to obtain multiple vial position recognition results; If it is determined that the plurality of level bubble position identification results all correspond to the level bubble being located at the center of the level bubble container, the current horizontal state of the energy storage product is taken as the second horizontal placement detection result.

[0057] In this embodiment, when identifying the positions of multiple vials in the second current scene image based on the multi-target detection model in the image recognition model, for example, Figure 3Taking the example of a level ruler 31 provided with three level bubble containers (and each level bubble container contains a level bubble), if the second camera in the above example captures the second current scene image corresponding to each level bubble container in the level ruler 31 from a side view angle, the second current scene image can be regarded as a front view angle image of the three level bubble containers. At this time, the positions of multiple level bubbles in the second current scene image can be identified based on the multi-target detection model in the image recognition model (refer to the above process of identifying the outer boundary contour of the energy storage product) to obtain multiple level bubble position identification results. Then, based on whether the multiple level bubble position identification results all correspond to the situation where the level bubble is located at the center of the level bubble container, the second horizontal placement detection result is finally determined.

[0058] In one embodiment, the server 10 is further specifically configured to: Detecting a plurality of vial container regions in the second current scene image based on a multi-target detection model in the image recognition model to obtain a plurality of vial container region detection results; If it is determined that a vial exists in the central area of ​​each of the multiple vial container area detection results, the vial located at the center of the vial container is used as the position identification result of each vial in the multiple vial position identification results.

[0059] In this embodiment, for example, after identifying the four vertices of the outer boundary contour of the level bubble container 1 among the three level bubble containers in the second current scene image, the container center point coordinate 1 of the level bubble container 1 can be determined; thereafter, after determining the four vertices of the outer boundary contour of the level bubble 1 in the level bubble container 1, the level bubble center point coordinate 1 can be determined; if it is determined that the container center point coordinate 1 coincides with the level bubble center point coordinate 1, it means that the level bubble 1 is located at the center position of the level bubble container 1. The judgment process of whether the level bubbles in other multiple level bubble containers are at the center position of the level bubble container is also referred to the above process. After obtaining the judgment results of whether the level bubbles in multiple level bubble containers are at the center position of the level bubble container, it can be comprehensively used to determine whether the energy storage product is currently in a horizontal state. Specifically, if it is determined that the multiple level bubble position recognition results all correspond to the level bubble being located at the center position of the level bubble container, the current horizontal state of the energy storage product is taken as the second horizontal placement detection result.

[0060] In one embodiment, the server 10 is further specifically configured to: If it is determined that among the multiple vial position identification results, there is at least one vial that is not located at the center of the vial container, the current non-horizontal state of the energy storage product is taken as the second horizontal placement detection result.

[0061] In this embodiment, if it is determined that among the multiple level bubble position identification results, there is at least one level bubble that is not located at the center of the level bubble container, it means that the horizontal fixing device of the energy storage product is in a non-horizontal state, and causes the energy storage product fixed thereon to be in a non-horizontal state. At this time, the current non-horizontal state of the energy storage product is used as the second horizontal placement detection result.

[0062] The server 10 is further configured to generate energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction, and send the energy storage product horizontal placement detection data to the user terminal if it is determined that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, and it is determined that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results.

[0063] In this embodiment, if it is determined in the server that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, it means that the energy storage product initially placed on the horizontal fixing device of the energy storage product has not been removed or illegally disassembled. Moreover, at this time, it is also determined that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results, which means that the energy storage product placed on the horizontal fixing device of the energy storage product is currently still in a horizontal placement state. At this time, energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction is generated, and the energy storage product horizontal placement detection data is sent to the user terminal. After receiving the energy storage product horizontal placement detection data, the user terminal can intuitively view the horizontal placement state of the energy storage product.

[0064] In one embodiment, the server 10 is specifically used for: The product name corresponding to the energy storage product fixed on the energy storage product horizontal fixing device, the product deployment position, the horizontal detection time corresponding to the energy storage product horizontal detection instruction, the first horizontal placement detection result and the second horizontal placement detection result are obtained, and filled into a preset detection data template to generate the energy storage product horizontal placement detection data.

[0065] In this embodiment, the server determines the specific product information (such as product name, product model, etc.) of the energy storage product placed on the energy storage product horizontal fixing device based on the energy storage product identification result, the first horizontal placement detection result, and the second horizontal placement detection result, and after determining that it is in a horizontal placement state, it can also obtain more specifically the product deployment position and the horizontal detection time corresponding to the energy storage product horizontal detection instruction, and finally fill the above information into the preset detection data template to generate the energy storage product horizontal placement detection data. By viewing the energy storage product horizontal placement detection data, the user terminal can obtain information such as the horizontal placement state of the energy storage product.

[0066] In one embodiment, the server 10 is further used for: If it is determined that the energy storage product identification result does not match the energy storage product information uploaded by the user terminal, or it is determined that the first horizontal placement detection result is a non-horizontal placement result, or it is determined that the second horizontal placement detection result is a non-horizontal placement result, energy storage product abnormal prompt data corresponding to the energy storage product horizontal detection instruction is generated, and the energy storage product abnormal prompt data is sent to the user terminal.

[0067] In this embodiment, if it is determined that the energy storage product identification result does not match the energy storage product information uploaded by the user terminal, it means that the energy storage product on the horizontal fixing device of the energy storage product has been moved or the equipment has been replaced; or if it is determined that the first horizontal placement detection result is a non-horizontal placement result, or if it is determined that the second horizontal placement detection result is a non-horizontal placement result, it means that the energy storage product on the horizontal fixing device of the energy storage product is not in a horizontal placement state. After any one of the above judgment conditions is met, it can be regarded that an abnormal situation has occurred in the energy storage product. At this time, the energy storage product abnormal prompt data corresponding to the energy storage product horizontal detection instruction is directly generated, and the energy storage product abnormal prompt data is sent to the user terminal. For example, the energy storage product abnormal prompt data includes the product name, product deployment location, the horizontal detection time corresponding to the energy storage product horizontal detection instruction, the first horizontal placement detection result and the second horizontal placement detection result. The above data is filled into the preset abnormal data prompt template to generate the energy storage product abnormal prompt data. The user terminal can obtain information such as the horizontal placement state of the energy storage product by viewing the energy storage product abnormal prompt data, and then determine whether to go to the site in time to handle the abnormal situation related to the energy storage product.

[0068] It can be seen that the implementation example of the system can perform remote unmanned detection of the horizontal placement state of the energy storage product based on the image captured by the image acquisition device of the energy storage product on the horizontal fixing device of the energy storage product, thereby improving the efficiency of obtaining the horizontal placement state of the energy storage product.

[0069] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0070] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0071] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0073] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting horizontal fixation of energy storage products, applied to a horizontal fixation detection system for energy storage products, characterized in that: The energy storage product horizontal fixation detection system comprises a server, an image acquisition device, an energy storage product horizontal fixation device and a user terminal, wherein the image acquisition device and the user terminal are both communicatively connected to the server, and the energy storage product horizontal fixation device is provided with a level ruler; the energy storage product horizontal fixation detection method comprises: The user terminal generates a field image acquisition instruction in response to the energy storage product level detection instruction and sends the instruction to the server, so that the server sends the field image acquisition instruction to the image acquisition device; The image acquisition device acquires images of the energy storage product horizontal fixing device and the energy storage product fixed on the energy storage product horizontal fixing device based on the on-site image acquisition instruction to obtain a first current on-site image, and sends the first current on-site image to the server; The image acquisition device acquires an image of the level ruler on the horizontal fixing device of the energy storage product to obtain a second current scene image, and sends the second current scene image to the server; The server performs image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and performs target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result; If the server determines that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, and determines that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results, it generates energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction, and sends the energy storage product horizontal placement detection data to the user terminal.

2. The method according to claim 1, characterized in that The performing image recognition on the first current scene image based on the pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result includes: Performing energy storage product distribution area detection on the first current scene image based on a multi-target detection model corresponding to the image recognition model to obtain a distribution target area of ​​energy storage products; Identify the product image corresponding to the distribution target area based on the multi-target detection model to obtain the energy storage product identification result; The boundary contour corresponding to the distributed target area is detected based on the multi-target detection model to obtain the first horizontal placement detection result.

3. The method according to claim 2, characterized in that The performing target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result includes: Identifying multiple vial positions in the second current scene image based on a multi-target detection model in the image recognition model to obtain multiple vial position recognition results; If it is determined that the plurality of level bubble position identification results all correspond to the level bubble being located at the center of the level bubble container, the current horizontal state of the energy storage product is taken as the second horizontal placement detection result.

4. The method according to claim 3, characterized in that After the step of identifying the positions of multiple vials in the second current scene image based on the multi-target detection model in the image recognition model to obtain multiple vial position recognition results, the method further includes: If it is determined that among the multiple vial position identification results, there is at least one vial that is not located at the center of the vial container, the current non-horizontal state of the energy storage product is taken as the second horizontal placement detection result.

5. The method according to claim 3, characterized in that: The identifying of the multiple vial positions in the second current scene image based on the multi-target detection model in the image recognition model to obtain multiple vial position identification results includes: Detecting a plurality of vial bubble container areas in the second current scene image based on a multi-target detection model in the image recognition model to obtain a plurality of vial bubble container area detection results; If it is determined that a vial exists in the central area of ​​each of the multiple vial container area detection results, the vial located at the center of the vial container is used as the position identification result of each vial in the multiple vial position identification results.

6. The method according to claim 1, characterized in that The generating of energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction includes: The product name corresponding to the energy storage product fixed on the energy storage product horizontal fixing device, the product deployment position, the horizontal detection time corresponding to the energy storage product horizontal detection instruction, the first horizontal placement detection result and the second horizontal placement detection result are obtained, and filled into a preset detection data template to generate the energy storage product horizontal placement detection data.

7. The method according to claim 1, characterized in that After the server performs image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and performs target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result, the method further includes: If the server determines that the energy storage product identification result does not match the energy storage product information uploaded by the user terminal, or determines that the first horizontal placement detection result is a non-horizontal placement result, or determines that the second horizontal placement detection result is a non-horizontal placement result, it generates energy storage product abnormal prompt data corresponding to the energy storage product horizontal detection instruction, and sends the energy storage product abnormal prompt data to the user terminal.

8. A horizontal fixing detection system for energy storage products, characterized in that: The energy storage product horizontal fixation detection system comprises a server, an image acquisition device, an energy storage product horizontal fixation device and a user terminal, wherein the image acquisition device and the user terminal are both communicatively connected to the server, and a level ruler is provided on the energy storage product horizontal fixation device; The user terminal is used to generate a field image acquisition instruction in response to the energy storage product level detection instruction and send it to the server, so that the server sends the field image acquisition instruction to the image acquisition device; The image acquisition device is used to acquire images of the energy storage product horizontal fixing device and the energy storage product fixed on the energy storage product horizontal fixing device based on the on-site image acquisition instruction to obtain a first current on-site image, and send the first current on-site image to the server; The image acquisition device is further used to acquire an image of the level ruler on the horizontal fixing device of the energy storage product to obtain a second current scene image, and send the second current scene image to the server; The server is configured to perform image recognition on the first current scene image based on a pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result, and perform target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result; The server is further configured to generate energy storage product horizontal placement detection data corresponding to the energy storage product horizontal detection instruction, and send the energy storage product horizontal placement detection data to the user terminal if it is determined that the energy storage product identification result matches the energy storage product information uploaded by the user terminal, and if it is determined that the first horizontal placement detection result and the second horizontal placement detection result are both horizontal placement results.

9. The energy storage product horizontal fixation detection system according to claim 8, characterized in that: The performing image recognition on the first current scene image based on the pre-trained image recognition model to obtain an energy storage product recognition result and a first horizontal placement detection result includes: Performing energy storage product distribution area detection on the first current scene image based on a multi-target detection model corresponding to the image recognition model to obtain a distribution target area of ​​energy storage products; Identify the product image corresponding to the distribution target area based on the multi-target detection model to obtain the energy storage product identification result; The boundary contour corresponding to the distributed target area is detected based on the multi-target detection model to obtain the first horizontal placement detection result.

10. The energy storage product horizontal fixation detection system according to claim 9, characterized in that: The performing target detection on the second current scene image based on the image recognition model to obtain a second horizontal placement detection result includes: Identifying multiple vial positions in the second current scene image based on a multi-target detection model in the image recognition model to obtain multiple vial position recognition results; If it is determined that the plurality of level bubble position identification results all correspond to the level bubble being located at the center of the level bubble container, the current horizontal state of the energy storage product is taken as the second horizontal placement detection result.

Images