Image acquisition method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image acquisition, and discloses an image acquisition method and device, electronic equipment and a storage medium. The image acquisition method is applied to inspection equipment and comprises the steps that inspection line information is read, and the inspection line information comprises point information of at least one inspection point; operation is carried out based on the inspection line information, when the inspection equipment operates to an inspection point, image acquisition is carried out on the inspection object at the inspection point, and a detection image of the inspection object is obtained; detecting the detection image of the inspection object to obtain a detection result of the detection image; and adjusting shooting parameters of the inspection equipment based on a detection result of the detection image, performing image acquisition on the inspection object again based on the adjusted shooting parameters, and updating the detection image of the inspection object. And the accuracy of inspection object image acquisition is improved.

Description

Technical Field

[0001] The present invention relates to the field of image acquisition technology, and in particular to an image acquisition method, device, electronic equipment and storage medium. Background Art

[0002] Image acquisition methods are widely used in equipment inspection. UAV inspections of power equipment have improved the efficiency and safety of power equipment inspections. Power grid maintenance personnel can obtain inspection results of on-site power equipment in a timely manner by remotely calling UAVs. When UAVs are affected by the external environment (such as wind speed, light intensity), UAV attitude deviation, or inaccurate route parameters (such as inaccurate shooting angle and focal length) during equipment inspections, it is easy to cause a large number of poor-quality photos such as object deviation, underexposure, and focus failure in the UAV inspection results, making it impossible for the intelligent algorithm deployed on the back end to correctly output recognition results. Summary of the invention

[0003] The present invention provides an image acquisition method, device, electronic equipment and storage medium to solve the problem of inaccurate image acquisition of inspection equipment.

[0004] According to one aspect of the present invention, there is provided an image acquisition method, the method comprising:

[0005] Reading inspection line information, where the inspection line information includes point information of at least one inspection point;

[0006] The inspection is performed based on the inspection line information. When the inspection equipment runs to the inspection point, the inspection object is imaged at the inspection point to obtain the inspection image of the inspection object.

[0007] Detecting the detection image of the inspection object to obtain the detection result of the detection image;

[0008] The shooting parameters of the inspection equipment are adjusted based on the detection result of the detection image, and the image of the inspection object is recaptured based on the adjusted shooting parameters to update the detection image of the inspection object.

[0009] According to another aspect of the present invention, there is provided an image acquisition device, the device comprising:

[0010] A patrol line information reading module, used to read patrol line information, where the patrol line information includes point information of at least one patrol point;

[0011] The detection image acquisition module is used to operate based on the inspection line information. When the inspection equipment runs to the inspection point, the inspection object is imaged at the inspection point to obtain the detection image of the inspection object;

[0012] The detection result acquisition module is used to detect the detection image of the inspection object and obtain the detection result of the detection image;

[0013] The detection image update module is used to adjust the shooting parameters of the inspection equipment based on the detection result of the detection image, and re-capture the image of the inspection object based on the adjusted shooting parameters to update the detection image of the inspection object.

[0014] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the image acquisition method of any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image acquisition method of any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention reads the inspection line information and operates based on the inspection line information. When the inspection equipment runs to the inspection point, the inspection object is imaged at the inspection point to obtain a detection image of the inspection object, and the detection image of the inspection object is detected to obtain a detection result of the detection image. The shooting parameters of the inspection equipment are adjusted based on the detection result of the detection image, and the image of the inspection object is re-captured based on the adjusted shooting parameters to update the detection image of the inspection object, thereby solving the problem of inaccurate image acquisition of the inspection equipment and improving the accuracy of image acquisition of the inspection object.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1is a flow chart of an image acquisition method provided by Embodiment 1 of the present invention;

[0023] Figure 2 is a structural schematic diagram of an image acquisition device provided by Embodiment 2 of the present invention;

[0024] Figure 3 It is a structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1

[0028] Figure 1This is a flow chart of an image acquisition method provided in the first embodiment of the present invention. This embodiment can be applied to the case where the shooting parameters of the inspection equipment are adjusted through the image captured from the inspection object to improve the detection accuracy of the detection image of the inspection object. The method can be executed by an image acquisition device, and the image acquisition device can be implemented in the form of hardware and / or software. The image acquisition device can be configured in the electronic device provided in the embodiment of the present invention. The electronic device can be an inspection device, such as a drone, an inspection robot, etc. The electronic device can be different in different inspection scenarios. The electronic device can be a drone or an inspection robot. Different inspection devices can be used in different inspection scenarios. The corresponding inspection device can be determined according to the inspection requirements. The above-mentioned inspection device can be configured with an image acquisition component, such as a camera, to capture images of the inspection object during the inspection process to obtain a detection image. For example, in the power inspection scenario, the inspection device can be a drone. Figure 1 As shown, the method includes:

[0029] S110: Read inspection line information, where the inspection line information includes point information of at least one inspection point.

[0030] In this embodiment, the inspection device may be a device for performing inspections, such as a drone, an inspection robot, etc. The inspection route is a pre-configured route, which includes the location points that the inspection device needs to pass through. The inspection device runs in sequence according to the location points in the inspection route. During the operation, the inspection object is inspected. For example, the inspection image of the inspection object can be obtained by capturing an image of the inspection object, and the inspection image is locally stored or transmitted to a related device. The related device can be understood as a device that detects abnormalities of the inspection object through a detection image, such as a computer or a mobile terminal, which detects abnormalities of the inspection object through a detection image to ensure the normal operation and safe use of the inspection object.

[0031] The inspection object may be equipment or products that need to be inspected. For example, in the inspection of power equipment, the inspection equipment may be a drone, and the inspection object may include power equipment, including but not limited to meters, disconnectors, circuit breakers, and transformers. The drone is used to inspect the power equipment according to the pre-set inspection route. For example, in industrial inspections, the inspection equipment may be an inspection robot, and the inspection object may be an industrial product that needs to be inspected. The inspection robot can be used to inspect the industrial product according to the established inspection route.

[0032] The inspection line is a line set in advance, which can be represented by the inspection line information. The inspection line information can be used to describe the inspection line, for example, it can be three-dimensional coordinate information including multiple position points on the inspection line. Optionally, the inspection line information can include point information of multiple inspection points, and the inspection point can be a position point that needs to be inspected in the inspection line. Each inspection point can correspond to an inspection object. For example, a certain inspection point in the inspection line of the power system can be a position point for inspecting a certain transformer. The point information of the inspection point can be information required to complete the inspection of the inspection object, for example, it can include three-dimensional coordinate information, action instructions, steering angles and ledger information of the inspection object, wherein the three-dimensional coordinate information is the position information of the inspection device at the inspection point; the ledger information of the inspection object can include a unique identification code (Identification, ID) of the inspection object; the action instruction can be a command for guiding the inspection device to complete the inspection action, for example, it can include but is not limited to shooting, hovering and steering. The steering angle can be the angle that the inspection device needs to adjust in order to meet the image acquisition of the inspection object at the inspection point, so that the inspection object can fall within the image acquisition range of the inspection device.

[0033] The inspection equipment can read the inspection line information through the storage system or wireless communication. By reading the inspection line information and parsing it, the point information of each inspection point in the inspection line can be accurately grasped, which helps the inspection equipment to accurately locate the inspection point. The parsed inspection line information can be stored in the inspection equipment for subsequent use.

[0034] In order to improve the accuracy and comprehensiveness of the inspection line information, auxiliary points are set in the inspection line, and accordingly, the auxiliary points and the inspection points form a complete inspection line, wherein the auxiliary points and the inspection points can be set according to the inspection requirements. Optionally, the inspection line information also includes point information of multiple auxiliary points, and multiple auxiliary points and at least one inspection point form an inspection line.

[0035] Specifically, the auxiliary point may be a position point that the inspection equipment passes through during the inspection process. The point information of the auxiliary point may be the position information of the auxiliary point, for example, the three-dimensional coordinate information of the auxiliary point. A complete inspection route may include all inspection points and all auxiliary points on the inspection route. There is a time sequence relationship between at least one auxiliary point and at least one inspection point in the inspection route information, and the inspection route information may be in the form of a point information sequence, that is, the point information of the auxiliary point and the point information of the inspection point are stored in a sequence. Each point information is read in sequence based on the point information sequence, and the inspection process is performed based on each point information read. The point information of the auxiliary point includes the position information of the auxiliary point; the point information of the inspection point includes the position information, action instructions and shooting parameters of the inspection point.

[0036] Specifically, the position information of the auxiliary point may be the coordinate information of the auxiliary point, for example, the three-dimensional coordinate information of the auxiliary point. The position information of the inspection point may be the coordinate information of the inspection point, for example, the three-dimensional coordinate information of the inspection point. The action instruction may be an instruction for guiding the inspection device to perform an action at the inspection point, for example, the action instruction may be shooting, hovering, turning, etc. The shooting parameter may be a parameter that affects the shooting effect of the inspection device, for example, the shooting parameter includes one or more of the shooting angle and the lighting information.

[0037] Specifically, the shooting angle can be the position and angle of the camera when the inspection equipment is shooting. In the case of drone inspection of power equipment, it can be the position and angle of the camera relative to the photographed power equipment when the drone is shooting. Lighting information can be various factors that affect the lighting conditions of the inspection equipment, such as light intensity, light direction, and light angle.

[0038] During the operation of the inspection equipment, the next point information in the inspection line information is read. If the next point information is the point information of the auxiliary point, that is, it includes three-dimensional coordinate information, the inspection equipment is controlled to continue to operate based on the three-dimensional coordinate information. If the next point information is the point information of the inspection point, the inspection equipment is controlled to continue to operate to the inspection point based on the three-dimensional coordinate information in the point information of the inspection point, and the inspection object is inspected at the inspection point based on the action instructions and shooting parameters. After the inspection is completed at the inspection point, the next point information is required and the inspection process continues.

[0039] S120: Run based on the inspection line information, and when the inspection equipment runs to the inspection point, collect images of the inspection object at the inspection point to obtain a detection image of the inspection object.

[0040] In this embodiment, the inspection object may be a device or product that needs to be inspected in the inspection line. In the inspection of power equipment, the inspection object may include but is not limited to transformers, circuit breakers, and disconnectors. The inspection device reads and analyzes the inspection line information, runs along the inspection line, and runs to the inspection points in sequence.

[0041] When the inspection device runs to the inspection point, it collects images of the inspection object at the inspection point to obtain the inspection image of the inspection object. When the inspection device does not run to the inspection point, the inspection device will not collect images. Only when the inspection device runs to the inspection point will the inspection device collect images of the inspection object at the inspection point, avoiding the collection of interfering images and reducing the energy consumption of the inspection device.

[0042] Optionally, the inspection device is controlled to operate based on the position information of the auxiliary point. When the inspection device runs to the inspection point, the action instructions in the point information of the inspection point are executed, and the inspection image of the inspection object is collected based on the shooting parameters.

[0043] When the inspection equipment runs to the inspection point, it performs corresponding actions according to the action instructions in the point information of the inspection point, collects images of the inspection object according to the shooting parameters, and obtains the detection image of the inspection object. For example, in the power system inspection, the inspection equipment is a drone. When the drone runs to a inspection point, the action instructions in the point information of the inspection point may include hovering instructions and shooting instructions. The drone executes the hovering instructions and shooting instructions in sequence at the inspection point, collects images of the inspection object, and obtains the detection image of the inspection object. Among them, the drone can execute the shooting instructions based on the shooting parameters, for example, based on the steering angle in the shooting parameters, adjust the lighting conditions during the shooting process based on the lighting information, execute shooting, and collect the detection image.

[0044] In some embodiments, the inspection equipment may be a drone, which obtains the current position and attitude information of the drone through navigation (such as a navigation system, gyroscope, accelerometer, etc.) and positioning (such as a global positioning system, real-time dynamic technology, visual positioning, etc.) sensors, and calculates the expected flight trajectory and attitude of the drone based on the point information of the auxiliary points and the point information of the inspection points, and sends instructions to a PID (Proportional-Integral-Derivative) controller to adjust the output of the drone to ensure that the drone runs along the inspection route and maintains a stable attitude.

[0045] S130: Detect the detection image of the inspection object to obtain a detection result of the detection image.

[0046] In order to improve the accuracy of the detection of the detection image of the inspection object collected, the detection image of the inspection object can be detected. In this embodiment, the detection result can be whether the detection image meets the image acquisition condition. It can be understood that by acquiring the detection image of the inspection object, the detection image of the inspection object is analyzed to obtain the analysis result of the inspection object, and the analysis result can be whether the inspection object is in a normal working state, whether there is an abnormality, etc. In order to avoid interference of the detection image with the analysis process, it is necessary to ensure that the detection image collected meets the image acquisition condition during the inspection process.

[0047] The image acquisition condition may include one or more of the position condition of the inspection object in the detection image and the image quality condition. Accordingly, detecting the detection image of the inspection object may include performing target detection and / or image quality detection on the detection image, and the detection result of the detection image may include one or more of the target detection result and the image quality detection result.

[0048] The image quality condition is a condition for determining the quality of the detection image. In some embodiments, the detection image can be quality-checked by a pre-set quality detection model to obtain a quality detection result. For example, the quality detection result can be represented by a value between 0 and 1, and the larger the value, the better the quality of the detection image. If the value of the quality detection result is less than the preset threshold, it indicates that the image quality is poor and does not meet the image quality condition, and the detection image needs to be re-collected.

[0049] Optionally, the evaluation index of image quality may include one or more of the brightness information, clarity and whether the detection image is jittered, etc. of the detection image. Accordingly, the image quality detection result may include one or more of the brightness detection result, the clarity detection result and the jitter detection result. Brightness information can be used to characterize the brightness of the detection image. For example, it can be the brightness value of the detection image. For example, if the brightness value of the captured detection image is within a preset brightness value range, it means that the captured detection image meets the image quality conditions. Clarity can be used to characterize whether the captured detection image is clear. For example, the inspection object in the detection image can be clearly identified, which means that the detection image meets the image quality conditions. During the shooting process, the detection image may be jittered due to factors such as the instability of the inspection equipment, which may cause the captured image to fail to meet the image quality conditions.

[0050] In some embodiments, the image quality of the detection image can be characterized by brightness information. Accordingly, the image quality detection of the detection image can include brightness detection of the detection image, and the image quality detection result includes a brightness detection result. Specifically, by extracting the brightness information of the detection image, the brightness detection result of the detection image is obtained based on the brightness information of the detection image.

[0051] Specifically, the brightness information can be used to characterize the brightness of the detection image, for example, it can be the brightness value of the detection image, and the brightness detection result can be used to characterize whether the brightness information of the detection image meets the conditions. For example, if the brightness value of the detection image exceeds the preset maximum brightness value, it means that the brightness of the detection image is too bright, and the brightness detection result may be that the brightness of the detection image is too bright and does not meet the conditions; if the brightness value of the detection image is lower than the preset minimum brightness value, it means that the brightness of the detection image is too dark, and the brightness detection result may be that the brightness of the detection image is too dark and does not meet the conditions; if the brightness value of the detection image is between the preset minimum brightness value and the maximum brightness value, it means that the brightness of the detection image is normal and meets the conditions.

[0052] In some embodiments, the detection image can also be detected by a pre-set clarity determination module and a jitter determination module to obtain a clarity detection result and a jitter detection result. The clarity detection result can be whether the detection image is clear, and the jitter detection result can be whether the detection image is jittered. When the detection image is not clear or jittered, it indicates that the detection image does not meet the image quality conditions and needs to be re-captured.

[0053] The position condition of the detection image is a condition for determining the area where the inspection object is located in the detection image. For example, the position condition of the detection image may be that the inspection object is located in the middle area of ​​the detection image. If the area where the inspection object is located in the detection image is located at the edge of the detection image or the inspection object is incomplete in the inspection image, it means that the detection image does not meet the position condition.

[0054] Identify the area where the inspection object is located in the inspection image, and determine whether the area where the inspection object is located in the inspection image meets the position condition of the inspection image. Optionally, target detection can be performed on the inspection image based on the target detection model to obtain the target detection result of the inspection image.

[0055] Specifically, the target detection result may be a result obtained by performing target detection on the detection image according to the target detection model.

[0056] In the above embodiment, a target detection model is used to detect the detection image of the inspection object. The target detection model can be a Transformer model and a convolutional neural network model, which can be set according to the needs of the user and is not limited here. Optionally, the target detection model can be a YOLOv3 model. The YOLOv3 model is a deep learning model that can realize fast and accurate detection of the detection image of the inspection object. The YOLOv3 model can learn the relevant features of the detection image through multi-layer convolution, activation and pooling operations, and detect the detection image of the inspection object.

[0057] Optionally, the training method of the target detection model includes: obtaining a sample set, the sample set including sample images of the inspection object and label information of the sample images, the label information including boundary information and / or preset boundary point information of the inspection object in the sample image; inputting the sample images of the inspection object into the target detection model to be trained to obtain predicted detection information; generating a first loss function based on the boundary information in the predicted detection information and the boundary information of the inspection object in the label information, and / or determining the predicted boundary point information based on the predicted detection information, and generating a second loss function based on the predicted boundary point information and the preset boundary point information in the label information; and adjusting the model parameters of the target detection model to be trained based on the first loss function and / or the second loss function.

[0058] By pre-creating a target detection model to be trained, initializing the model parameters of the target detection model to be trained, and training the initialized target detection model based on the sample set, a trained target detection model is obtained. Before training the target detection model, the sample set needs to be pre-processed, for example, the size of the sample image of the inspection object can be adjusted to the size required by the target detection model, for example, the size of the sample image of the inspection object can be adjusted to 416×416.

[0059] The sample set may be a set of image data of the inspection objects in the inspection route, which may be obtained by performing image acquisition on the inspection objects, or by reading the image data of the relevant inspection objects from a relevant database, in which the image data of multiple inspection objects are stored, and the inspection object identifier of the inspection object may be used for matching to the sample set. The sample image may be an image of the inspection object. The label information may be the boundary information and / or preset boundary point information of the inspection object in the sample image, for example, the boundary information may be the boundary box and boundary coordinates of the inspection object in the sample image. The label information may be the boundary coordinates of the inspection object in the sample image, and the boundary coordinates may be determined based on the upper left corner coordinates and the lower right corner coordinates of the inspection object in the sample image, for example, the upper left corner coordinates and the lower right corner coordinates of the inspection object in the sample image are (x 1 ,y 1 ) and (x 2 ,y 2 ), the boundary coordinates are (x 1 ,y 1 , x 2 ,y 2 ). The label information may also be the preset boundary point information of the inspection object in the sample image. The preset boundary point information may be the boundary box and boundary coordinates of the inspection object preset in the sample image. The label information may also be the boundary information and preset boundary point information of the inspection object in the sample image. The predicted detection information may be the boundary coordinates of the inspection object predicted by the target detection model to be trained, for example, the predicted upper left corner coordinates and lower right corner coordinates of the inspection object.

[0060] The first loss function can be used to characterize the error between the boundary information in the predicted detection information and the boundary information of the inspection object in the label information. For example, it can be the average of the errors between the boundary information in the predicted detection information of multiple inspection objects and the boundary information of the inspection object in the label information. The predicted boundary point information can be the boundary coordinates of the inspection object predicted by the target detection model to be trained, which can be obtained based on the predicted detection information. For example, the predicted detection information is the upper left corner coordinates (x n ,y n ) and the lower right corner coordinate (x n ,y n), the predicted boundary point information is obtained based on the predicted detection information as (x n ,y n , x n ,y n ). The second loss function can be used to characterize the error between the predicted boundary point information and the preset boundary point information in the label information, for example, it can be the average of the difference between the predicted boundary point information of multiple inspection objects and the preset boundary point information in the label information.

[0061] The model parameters of the target detection model to be trained can be adjusted according to the first loss function, for example, the first loss function can be minimized. The model parameters of the target detection model to be trained can also be adjusted according to the second loss function, for example, the second loss function can be minimized. The model parameters of the target detection model to be trained can also be adjusted according to the first loss function and the second loss function, for example, the first loss function and the second loss function can be minimized so that the sum of the first loss function and the second loss function is minimized.

[0062] Optionally, the target detection result includes boundary information of at least one object. Since the shooting range of the inspection device may include the inspection object corresponding to the inspection point and other objects when collecting the inspection image of the inspection object, the inspection image may include the image content of at least one object, where the at least one object includes the inspection object. In order to reduce the interference of the boundary information of other objects on the inspection image, the boundary information of the inspection object is identified in the target detection result, and the boundary information of other objects is deleted.

[0063] Optionally, the inspection object identification is included in the point information of the inspection point. Specifically, the inspection object identification can be information used to identify and record the characteristics and status of the inspection object. For example, in the inspection of power equipment, the inspection object identification can be the unique ID of the transformer. Different inspection objects have different inspection object identifications, and the inspection object identification of each inspection object is unique.

[0064] By performing target detection on the inspection object, multiple boundary information can be identified, and the boundary information of the inspection object can be matched with the inspection object identifier to avoid interference from other boundary information.

[0065] Optionally, the method further includes: matching the boundary information of at least one inspection object in the target detection result corresponding to the inspection point with the inspection object identifier to determine the boundary information of the inspection object. By matching the boundary information in the target detection result corresponding to the inspection point with the inspection object identifier, the inspection object of the inspection point can be determined, thereby determining the boundary information of the inspection object, avoiding interference from other boundary information in target detection, and improving the accuracy of detection image detection of the inspection object.

[0066] In the above embodiment, the detection process of the detection image is executed by the detection task, and the detection task is triggered when the inspection device runs to the inspection point and ends when the inspection device leaves the inspection point, wherein the detection task can be used to detect the detection image, including target detection and / or image quality detection. During the inspection process of the inspection device, when the inspection device arrives at the inspection point, the detection task is started, and when the inspection device leaves the inspection point, the detection task is ended, thereby reducing the energy consumption of the inspection device. The inspection device leaving the inspection point indicates that the detection of the detection image has been completed, and the collected detection image meets the image collection conditions.

[0067] S140: adjusting the shooting parameters of the inspection device based on the detection result of the detection image, and re-capturing the image of the inspection object based on the adjusted shooting parameters to update the detection image of the inspection object.

[0068] In this embodiment, the shooting parameters of the inspection device are adjusted according to the detection result of the detection image. When the detection result of the detection image does not meet the image acquisition conditions, for example, the inspection object in the detection image is located in the upper right corner, it indicates that the detection image of the inspection object collected is inaccurate, and it is necessary to adjust the shooting angle of the inspection object, re-capture the image of the inspection object, and update the inspection image of the inspection object. For example, if the overall brightness of the detection image is too dark or the brightness of the area where the inspection object is located in the detection image is too dark, it indicates that the lighting information during the shooting process of the detection image is inaccurate, and it is necessary to adjust the lighting information during the shooting process.

[0069] It can be understood that when the detection result of the detection image does not meet one or more of the position conditions and image quality conditions of the detection image, it indicates that the detection result of the detection image does not meet the image acquisition conditions, and the shooting parameters need to be adjusted and the detection image of the inspection object needs to be re-captured.

[0070] Optionally, adjusting shooting parameters of the inspection device based on the detection results of the detection image includes: determining the relative position of the inspection object in the detection image based on boundary information of the inspection object in the target detection result, and adjusting the shooting angle based on the relative position of the inspection object in the detection image.

[0071] Adjusting the shooting parameters of the inspection device based on the detection results of the detection image may be to determine the relative position of the inspection object in the detection image according to the boundary information of the inspection object in the target detection result, and to adjust the shooting angle based on the relative position of the inspection object in the detection image. For example, if it is determined that the position of the inspection object in the detection image is slightly upward according to the boundary information of the inspection object in the target detection result, the shooting angle of the inspection device needs to be adjusted upward so that the inspection object is located in the middle of the detection image.

[0072] During the shooting process of the inspection equipment, the illumination information will affect the shooting effect of the detection image, and further affect the brightness detection result of the detection image. Therefore, in order to reduce the influence of the illumination information on the brightness detection result of the detection image, it is necessary to adjust the illumination information.

[0073] Optionally, adjusting the shooting parameters of the inspection device based on the detection result of the detection image includes: adjusting the lighting information based on the brightness detection result.

[0074] The shooting parameters of the inspection equipment can be adjusted based on the detection results of the detection image, and the lighting information can also be adjusted according to the brightness detection results. For example, if the brightness detection result shows that the exposure of the inspection image of the inspection object is too high, the exposure of the inspection equipment can be lowered.

[0075] Optionally, the shooting parameters of the inspection equipment are adjusted based on the detection results of the detection image, including: determining the relative position of the inspection object in the detection image based on the boundary information of the inspection object in the target detection result, adjusting the shooting angle based on the relative position of the inspection object in the detection image, and adjusting the lighting information based on the brightness detection result.

[0076] The shooting parameters of the inspection equipment can be adjusted based on the detection results of the detection image. The shooting angle can also be adjusted based on the boundary information of the inspection object in the target detection result to determine the relative position of the inspection object in the detection image, and the lighting information can be adjusted based on the brightness detection result.

[0077] Optionally, the method also includes: if the detection result of the detection image meets the image acquisition condition, controlling the inspection equipment to continue running along the inspection route information, wherein the image acquisition condition includes one or more of the position condition of the inspection object in the detection image and the image quality condition.

[0078] Specifically, the position condition may be a relative position condition that the inspection object needs to meet in the detection image, and the image quality condition may be a brightness condition that the detection image needs to meet. The image acquisition condition may be a condition that the inspection image of the inspection object needs to meet during the acquisition process, and may be one or more of the position condition and image quality condition of the inspection object in the detection image.

[0079] The technical solution of this embodiment reads the inspection route information and parses it, controls the inspection equipment to operate based on the inspection route information, and when the inspection equipment runs to the inspection point, performs image acquisition on the inspection object at the inspection point to obtain the detection image of the inspection object, pre-creates a target detection model to be trained, performs model parameter initialization processing on the target detection model to be trained, trains the initialized target detection model based on the sample set to obtain the trained target detection model, adjusts the shooting parameters of the inspection equipment based on the detection result of the detection image, and re-acquires the image of the inspection object based on the adjusted shooting parameters, updates the detection image of the inspection object, reduces the energy consumption of the inspection equipment, and improves the accuracy of image acquisition of the inspection object.

[0080] Embodiment 2

[0081] Figure 2 Schematic diagram of the structure of an image acquisition device provided by Embodiment 2 of the present invention. Figure 2 As shown, the device comprises:

[0082] The inspection line information reading module 210 is used to read the inspection line information, where the inspection line information includes point information of at least one inspection point;

[0083] The detection image acquisition module 220 is used to operate based on the inspection line information, and when the inspection equipment runs to the inspection point, the inspection object is imaged at the inspection point to obtain the detection image of the inspection object;

[0084] The detection result acquisition module 230 is used to detect the detection image of the inspection object and obtain the detection result of the detection image;

[0085] The detection image updating module 240 is used to adjust the shooting parameters of the inspection equipment based on the detection result of the detection image, and re-capture the image of the inspection object based on the adjusted shooting parameters to update the detection image of the inspection object.

[0086] On the basis of the above embodiment, optionally, the detection image acquisition module 220 is also used to control the inspection equipment to operate based on the position information of the auxiliary point. When the inspection equipment runs to the inspection point, the action instructions in the point information of the inspection point are executed, and the detection image of the inspection object is collected based on the shooting parameters.

[0087] Optionally, the detection result acquisition module 230 is also used for training the target detection model, and the training method of the target detection model includes: obtaining a sample set, the sample set including a sample image of the inspection object and label information of the sample image, the label information including boundary information and / or preset boundary point information of the inspection object in the sample image; inputting the sample image of the inspection object into the target detection model to be trained to obtain predicted detection information; generating a first loss function based on the boundary information in the predicted detection information and the boundary information of the inspection object in the label information, and / or determining the predicted boundary point information based on the predicted detection information, and generating a second loss function based on the predicted boundary point information and the preset boundary point information in the label information; and adjusting the model parameters of the target detection model to be trained based on the first loss function and / or the second loss function.

[0088] Optionally, the detection result acquisition module 230 is further used to perform target detection on the detection image based on the target detection model to obtain the target detection result of the detection image.

[0089] Optionally, the detection result acquisition module 230 is further used to extract brightness information of the detection image, and obtain a brightness detection result of the detection image based on the brightness information of the detection image.

[0090] Optionally, the detection result acquisition module 230 is also used to execute the detection process of the detection image through the detection task, and the detection task is triggered when the inspection device runs to the inspection point and ends when the inspection device leaves the inspection point.

[0091] Optionally, the detection result acquisition module 230 is also used to perform target detection on the detection image based on the target detection model to obtain the target detection result of the detection image; and, extract brightness information of the detection image, and obtain the brightness detection result of the detection image based on the brightness information of the detection image.

[0092] Optionally, the detection image updating module 240 is further used to determine the relative position of the inspection object in the detection image based on the boundary information of the inspection object in the target detection result, and adjust the shooting angle based on the relative position of the inspection object in the detection image.

[0093] Optionally, the detection image updating module 240 is further configured to adjust the illumination information based on the brightness detection result.

[0094] Optionally, the detection image update module 240 is also used to determine the relative position of the inspection object in the detection image based on the boundary information of the inspection object in the target detection result, adjust the shooting angle based on the relative position of the inspection object in the detection image, and adjust the lighting information based on the brightness detection result.

[0095] Optionally, the detection result acquisition module 230 is further used to match the boundary information of at least one object in the target detection result corresponding to the inspection point with the inspection object identifier to determine the boundary information of the inspection object.

[0096] Optionally, the device also includes a detection result judgment module, which is used to: if the detection result of the detection image meets the image acquisition condition, control the inspection equipment to continue running along the inspection route information, wherein the image acquisition condition includes one or more of the position condition of the inspection object in the detection image and the image quality condition.

[0097] The image acquisition device provided in the embodiment of the present invention can execute the image acquisition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0098] Embodiment 3

[0099] Figure 3 1 is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0100] like Figure 3 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0101] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as an image acquisition method.

[0103] In some embodiments, the image acquisition method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image acquisition method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image acquisition method in any other appropriate manner (e.g., by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] The computer programs for implementing the image acquisition method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] Embodiment 4

[0107] Embodiment 4 of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute an image acquisition method, the method comprising:

[0108] Read inspection route information, which includes point information of at least one inspection point; operate based on the inspection route information, and when the inspection equipment runs to the inspection point, capture an image of the inspection object at the inspection point to obtain a detection image of the inspection object; detect the detection image of the inspection object to obtain a detection result of the detection image; adjust the shooting parameters of the inspection equipment based on the detection result of the detection image, and re-capture the image of the inspection object based on the adjusted shooting parameters to update the detection image of the inspection object.

[0109] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0112] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0114] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image acquisition method, characterized in that: Applied to inspection equipment, including: Reading inspection line information, wherein the inspection line information includes point information of at least one inspection point; Based on the inspection line information, when the inspection device runs to the inspection point, an image of the inspection object is collected at the inspection point to obtain a detection image of the inspection object; Detecting the detection image of the inspection object to obtain a detection result of the detection image; The shooting parameters of the inspection device are adjusted based on the detection result of the detection image, and the image of the inspection object is recaptured based on the adjusted shooting parameters to update the detection image of the inspection object.

2. The method according to claim 1, characterized in that The inspection route information also includes point information of a plurality of auxiliary points, and the plurality of auxiliary points and the at least one inspection point form the inspection route; The point information of the auxiliary point includes the position information of the auxiliary point; The point information of the inspection point includes the location information, action instructions and shooting parameters of the inspection point; When the inspection device runs to the inspection point, an image of the inspection object is collected at the inspection point to obtain a detection image of the inspection object, including: The inspection device is controlled to operate based on the position information of the auxiliary point. When the inspection device operates to the inspection point, the action instructions in the point information of the inspection point are executed, and the inspection image of the inspection object is acquired based on the shooting parameters.

3. The method according to claim 1, characterized in that The detecting the detection image of the inspection object to obtain the detection result of the detection image includes: Performing target detection on the detection image based on the target detection model to obtain the target detection result of the detection image; and / or extracting brightness information of the detection image to obtain the brightness detection result of the detection image based on the brightness information of the detection image; The detection process of the detection image is executed through a detection task, and the detection task is triggered when the inspection device runs to the inspection point and ends when the inspection device leaves the inspection point.

4. The method according to claim 3, characterized in that The point information of the inspection point includes an inspection object identifier; the target detection result includes boundary information of at least one object; The method further includes: matching the boundary information of at least one object in the target detection result corresponding to the inspection point with the inspection object identifier to determine the boundary information of the inspection object.

5. The method according to claim 3, characterized in that: The training method of the target detection model includes: Acquire a sample set, wherein the sample set includes a sample image of the inspection object and label information of the sample image, wherein the label information includes boundary information and / or preset boundary point information of the inspection object in the sample image; Inputting the sample image of the inspection object into the target detection model to be trained to obtain predicted detection information; Generate a first loss function based on the boundary information in the predicted detection information and the boundary information of the inspection object in the label information, and / or determine the predicted boundary point information based on the predicted detection information, and generate a second loss function based on the predicted boundary point information and the preset boundary point information in the label information; The model parameters of the target detection model to be trained are adjusted based on the first loss function and / or the second loss function.

6. The method according to claim 3, characterized in that The shooting parameters include one or more of shooting angle and lighting information; The adjusting the shooting parameters of the inspection device based on the detection result of the detection image includes: Determine the relative position of the inspection object in the detection image based on the boundary information of the inspection object in the target detection result, and adjust the shooting angle based on the relative position of the inspection object in the detection image; And / or, adjusting the illumination information based on the brightness detection result.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: If the detection result of the detection image meets the image acquisition condition, the inspection device is controlled to continue running along the inspection route information, wherein the image acquisition condition includes one or more of the position condition of the inspection object in the detection image and the image quality condition.

8. An image acquisition device, characterized in that: include: A patrol line information reading module, used to read patrol line information, wherein the patrol line information includes point information of at least one patrol point; A detection image acquisition module is used to operate based on the inspection line information, and when the inspection equipment runs to the inspection point, the inspection object is imaged at the inspection point to obtain the detection image of the inspection object; A detection result acquisition module, used to detect the detection image of the inspection object and obtain the detection result of the detection image; The detection image updating module is used to adjust the shooting parameters of the inspection device based on the detection result of the detection image, and re-capture the image of the inspection object based on the adjusted shooting parameters to update the detection image of the inspection object.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the image acquisition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image acquisition method according to any one of claims 1 to 7 when executed.

Images