Image object distance fusion method, device, equipment and storage medium

Abstract

The present invention discloses an image object distance fusion method, device, equipment and storage medium, including: obtaining the distance measurement value between each target in the monitoring range and the preset position point in real time through a distance sensor; collecting the environmental image corresponding to the monitoring range according to the preset time interval, and identifying each target existing in the environmental image to obtain the image recognition result corresponding to each target; determining the predicted distance between each target in the environmental image and the preset position point according to the distance measurement value between each target and the preset position point, and the image recognition result corresponding to each target. The technical solution of the embodiment of the present invention can reduce the error of the image object distance fusion result and improve the accuracy of the image object distance fusion result.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image-object distance fusion method, device, equipment and storage medium. Background Art

[0002] At present, many visualization services need to collect multiple business-related images during implementation, and detect the target information of each target included in the image, such as distance, size, etc. By detecting the target information included in multiple images, multiple images can be fused to obtain a large scene image, which is of great significance to visualization services.

[0003] In the prior art, when detecting the distance information of a target in an image, the distance, size, etc. information of other targets is usually calculated based on the information (such as position, size, etc.) of a reference object in the image.

[0004] However, in the prior art, when the shooting environment changes, such as in windy and rainy weather, strong light and other environments, the authenticity of the reference object information in the image will be affected, thereby causing deviations in the target information; secondly, the distance information of the target in the prior art will have errors due to factors such as image resolution and brightness. Summary of the invention

[0005] The present invention provides an image-object distance fusion method, device, equipment and storage medium, which can reduce the error of image-object distance fusion results and improve the accuracy of image-object distance fusion results.

[0006] According to one aspect of the present invention, there is provided an image object distance fusion method, comprising:

[0007] Through the distance sensor, the distance measurement value between each target within the monitoring range and the preset position point is obtained in real time;

[0008] According to a preset time interval, an environmental image corresponding to the monitoring range is collected, and each target existing in the environmental image is identified to obtain an image recognition result corresponding to each target;

[0009] The predicted distance between each target and the preset position point in the environment image is determined according to the distance measurement value between each target and the preset position point and the image recognition result corresponding to each target.

[0010] According to another aspect of the present invention, there is provided an image-object distance fusion device, the device comprising:

[0011] The distance measurement module is used to obtain the distance measurement value between each target in the monitoring range and the preset position point in real time through the distance sensor;

[0012] The target recognition module is used to collect the environment image corresponding to the monitoring range according to the preset time interval, and recognize each target existing in the environment image to obtain the image recognition result corresponding to each target;

[0013] The predicted distance determination module is used to determine the predicted distance between each target in the environment image and the preset position point according to the distance measurement value between each target and the preset position point and the image recognition result corresponding to each target.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image-object distance fusion method described in any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image-object distance fusion method described in any embodiment of the present invention when executed.

[0019] The technical solution provided by the embodiment of the present invention uses a distance sensor to obtain the distance measurement value between each target in the monitoring range and the preset position point in real time, collects the environmental image corresponding to the monitoring range according to the preset time interval, and identifies each target existing in the environmental image to obtain the image recognition result corresponding to each target, and determines the predicted distance between each target in the environmental image and the preset position point according to the distance measurement value between each target and the preset position point, and the image recognition result corresponding to each target. The technical means can reduce the error of the image object distance fusion result and improve the accuracy of the image object distance fusion result.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a flow chart of an image object distance fusion method provided according to an embodiment of the present invention;

[0023] Figure 2 is a flow chart of another image object distance fusion method provided according to an embodiment of the present invention;

[0024] Figure 3 is a flow chart of another image object distance fusion method provided according to an embodiment of the present invention;

[0025] Figure 4 is a structural schematic diagram of an image-object distance fusion device provided according to an embodiment of the present invention;

[0026] Figure 5 It is a structural schematic diagram of an electronic device for implementing the image-object distance fusion method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Figure 1This is a flowchart of an image-object distance fusion method provided in the first embodiment of the present invention. This embodiment is applicable to the case where the target distance in the monitoring image is detected and the target distance is fused with the image. The method can be executed by an image-object distance fusion device. The image-object distance fusion device can be implemented in the form of hardware and / or software. The image-object distance fusion device can be configured in an electronic device with data processing function (such as a terminal or server). Figure 1 As shown, the method includes:

[0030] Step 110: Obtain the distance measurement value between each target within the monitoring range and the preset position point in real time through the distance sensor.

[0031] In this embodiment, the distance sensor is used to measure the distance between an object and a preset position point. Specifically, the distance sensor includes a laser distance sensor, an ultrasonic distance sensor, an infrared distance sensor, a radar distance sensor, etc. The monitoring range can be a preset range for monitoring the entry and exit of people or objects. Taking high-voltage wires as an example, the monitoring range can be a preset dangerous area around the high-voltage wires.

[0032] In this step, the distance sensor can be used to obtain in real time the distance measurement value between each target (including personnel or objects, etc.) within the monitoring range and a preset position point, and the preset position point can be the position point corresponding to the distance sensor.

[0033] Step 120: According to a preset time interval, an environmental image corresponding to the monitoring range is collected, and each target existing in the environmental image is identified to obtain an image recognition result corresponding to each target.

[0034] In this embodiment, while obtaining the distance measurement values ​​of each target within the monitoring range, an image acquisition device (such as a camera, etc.) is also used to collect images of the monitoring range (i.e., environmental images) at a preset time interval. The time interval may be 3s, and the specific value may be preset according to actual conditions, which is not limited in this embodiment.

[0035] In this step, after the environmental image is collected, each target in the environmental image can be identified according to the pre-trained image recognition model to obtain the classification information and location information of each target, and the classification information and location information of the target are used as the image recognition result.

[0036] In a specific embodiment, before identifying each target in an environmental image, multiple environmental sample images can be acquired in advance, and the targets in each environmental sample image can be labeled to obtain multiple training sample images. The multiple training sample images are then input into a neural network model, and the neural network model is iteratively trained to obtain the image recognition model.

[0037] Step 130: Determine the predicted distance between each target in the environment image and the preset position point according to the distance measurement value between each target and the preset position point and the image recognition result corresponding to each target.

[0038] In this step, optionally, the distance measurement value of each target can be calibrated according to the position information of each target output by the image recognition model, and the calibrated distance measurement value can be used as the predicted distance corresponding to the target. The predicted distance is then filled into the corresponding target in the environmental image to obtain the image object distance fusion result.

[0039] In a specific embodiment, it is assumed that there are two targets in the environment image, namely target A and target B. The distance measurement value of target A obtained by the distance sensor is equal to the distance measurement value of target B. The horizontal coordinate value corresponding to target A in the environment image information is equal to the horizontal coordinate value corresponding to target B in the environment image, but the vertical coordinate value corresponding to target A in the environment image information is not equal to the vertical coordinate value corresponding to target B in the environment image. Therefore, the respective distance measurement values ​​of target A and target B can be calibrated according to the vertical coordinate values ​​corresponding to each of them.

[0040] In this embodiment, by combining the hardware ranging sensor with the AI ​​algorithm of the image recognition model, there is no need to infer the distance of the target according to the reference object in the image, thereby avoiding changes in the shooting environment and affecting the authenticity of the reference object information in the image; secondly, by obtaining the distance measurement value of the target through the ranging sensor and calibrating the distance measurement value according to the image recognition result, it is possible to avoid errors in the image object distance fusion result caused by factors such as the resolution or brightness of the environmental image.

[0041] The technical solution provided by the embodiment of the present invention uses a distance sensor to obtain the distance measurement value between each target in the monitoring range and the preset position point in real time, collects the environmental image corresponding to the monitoring range according to the preset time interval, and identifies each target existing in the environmental image to obtain the image recognition result corresponding to each target, and determines the predicted distance between each target in the environmental image and the preset position point according to the distance measurement value between each target and the preset position point, and the image recognition result corresponding to each target. The technical means can reduce the error of the image object distance fusion result and improve the accuracy of the image object distance fusion result.

[0042] Figure 2 This is a flow chart of an image object distance fusion method provided in the second embodiment of the present invention. This embodiment is a further refinement of the above embodiment. Figure 2 As shown, the method includes:

[0043] Step 201: Obtain the distance measurement value between each target within the monitoring range and the preset position point in real time through the distance sensor.

[0044] Step 202: Obtain coordinate information corresponding to each target within the monitoring range in real time through the distance sensor.

[0045] In this step, optionally, the horizontal coordinate value and the vertical coordinate value corresponding to each target can be obtained through a distance sensor according to the two-dimensional coordinate system corresponding to the monitoring range.

[0046] Step 203: Arrange the targets in ascending order of horizontal coordinate values.

[0047] In this step, the targets within the monitoring range may be arranged in order of horizontal coordinate values ​​from small to large.

[0048] In a specific embodiment, assume that the monitoring range includes the following targets: target A, target B and target C, where the horizontal coordinate value of target A is 4, the horizontal coordinate value of target B is -6, and the horizontal coordinate value of target C is 11, then the arranged targets are: target B, target A and target C.

[0049] Step 204: Store the arranged targets, the horizontal coordinate values ​​corresponding to the targets, and the distance measurement values ​​in a distance list in sequence.

[0050] In this step, a distance list can be constructed according to the mapping relationship between the arranged targets, horizontal coordinate values, and distance measurement values, as shown in Table 1:

[0051] Table 1

[0052] Target ID Horizontal coordinate value Distance measurement 1 -6 12m 2 4 18m 3 11 9m …… …… ……

[0053] Step 205: Collect environmental images corresponding to the monitoring range according to a preset time interval.

[0054] Step 206: Identify the objects in the environment image and obtain size information corresponding to each object.

[0055] In this step, the size information of each target, such as the length and width of the target, can be identified through the image recognition model.

[0056] Step 207: Generate a picture frame corresponding to each target in the environment image according to the size information corresponding to each target.

[0057] In this embodiment, optionally, a picture frame corresponding to each target may be generated in the environment image according to size information of each target and boundary information corresponding to each target.

[0058] Step 208: In the distance list, in accordance with the arrangement order of the targets, the distance measurement values ​​corresponding to the targets are obtained in sequence, and the distance measurement values ​​corresponding to the targets are combined to obtain the image name of the environment image.

[0059] In this step, the distance measurement values ​​of each target may be obtained in sequence, and the distance measurement values ​​of each target may be combined according to a preset separator to obtain an image name.

[0060] In a specific embodiment, taking the distance list in Table 1 as an example, assuming that the distance list only includes target 1, target 2, and target 3, the generated image name may be “12m_18m_9m.jpg”.

[0061] Step 209: fill the distance measurement values ​​corresponding to the respective targets into the picture frames corresponding to the respective targets in sequence according to the image names of the environmental images.

[0062] In this step, the distance measurement value of each target can be extracted from the image name of the environment image according to the preset separator, and the distance measurement value of each target is filled into the corresponding picture frame.

[0063] Step 210: Determine the predicted distance between each target and a preset position point according to the distance filling result corresponding to each picture frame.

[0064] In this step, optionally, the distance filling result corresponding to each picture frame can be directly used as the predicted distance of the corresponding target.

[0065] In this embodiment, when the distance between two targets within the monitoring range is close, the distance sensor may only obtain the distance measurement value of one of the targets. In order to avoid omissions in the measurement values ​​obtained by the distance sensor, this embodiment provides a method of using an AI algorithm to identify targets and generate target frames, thereby accurately obtaining all targets within the monitoring range and obtaining the predicted distance of each target based on the positional relationship between the targets.

[0066] In one implementation of the present embodiment, the predicted distance between each of the targets and the preset position point is determined based on the distance filling results corresponding to each of the picture frames, including: if there are multiple picture frames corresponding to the same target horizontal coordinate values, then the target vertical coordinate values ​​corresponding to the multiple picture frames are obtained; according to the target vertical coordinate values ​​corresponding to each of the picture frames, the distance filling results corresponding to each of the picture frames are corrected to obtain the predicted distance between each of the targets and the preset position point.

[0067] In a specific embodiment, assuming that the horizontal coordinate values ​​of the two targets acquired by the distance sensor are equal, but the vertical coordinate values ​​are not equal, the vertical coordinate values ​​of the two targets can be arranged in order from small to large. The smaller the vertical coordinate value, the smaller the predicted distance of the corresponding target, and the larger the vertical coordinate value, the larger the predicted distance of the corresponding target. In the above manner, the distance filling results corresponding to each picture frame can be calibrated to obtain the predicted distance corresponding to each target.

[0068] The technical solution provided by the embodiment of the present invention uses a distance sensor to obtain the distance measurement value of each target within the monitoring range in real time, uses the distance sensor to obtain the coordinate information corresponding to each target within the monitoring range in real time, arranges each target in the order of the horizontal coordinate value from small to large, and sequentially stores each arranged target, the horizontal coordinate value corresponding to each target, and the distance measurement value into a distance list, collects an environmental image corresponding to the monitoring range according to a preset time interval, identifies the target existing in the environmental image to obtain the target size information, generates a target picture frame according to the size information corresponding to each target, sequentially obtains the distance measurement value corresponding to each target in the distance list according to the arrangement order of the target, and combines the distance measurement value corresponding to each target to obtain an image name, and sequentially fills the distance measurement value of each target into the picture frame corresponding to each target according to the image name, and determines the technical means of predicting the distance of each target according to the distance filling result corresponding to each picture frame, which can reduce the error of the image object distance fusion result and improve the accuracy of the image object distance fusion result.

[0069] Figure 3 This is a flow chart of an image object distance fusion method provided in Embodiment 3 of the present invention. This embodiment is a further refinement of the above embodiment. Figure 3 As shown, the method includes:

[0070] Step 301: Obtain the distance measurement value between each target within the monitoring range and the preset position point in real time through the distance sensor.

[0071] Step 302: Identify the objects in the environment image and obtain size information corresponding to each object.

[0072] Step 303: Generate a picture frame corresponding to each target in the environment image according to the size information corresponding to each target.

[0073] Step 304: in the distance list, in accordance with the arrangement order of the targets, the distance measurement values ​​corresponding to the targets are obtained in sequence, and the distance measurement values ​​corresponding to the targets are combined to obtain the image name of the environment image.

[0074] Step 305: fill the distance measurement values ​​corresponding to the respective targets into the picture frames corresponding to the respective targets in sequence according to the image names of the environmental images.

[0075] Step 306: Determine the predicted distance between each target and a preset position point according to the distance filling result corresponding to each picture frame.

[0076] Step 307: Sort the picture frames according to their corresponding areas.

[0077] In this step, the picture frames may be optionally sorted in order of area from small to large.

[0078] Step 308: Determine a reference distance between each target and a preset position point according to the sorted picture frames and the target prediction distance corresponding to each picture frame.

[0079] In this embodiment, the target predicted distance is the predicted distance between the target corresponding to the picture frame and the preset position point. The reference distance may be the distance after the predicted distance is calibrated.

[0080] In a specific embodiment, the larger the area of ​​the picture frame, the smaller the actual distance between the corresponding target and the preset position point is, and the smaller the area of ​​the picture frame, the larger the actual distance between the corresponding target and the preset position point is. Therefore, in the above manner, the target prediction distance of each picture frame can be calibrated according to the sorted picture frames to obtain the reference distance of each target.

[0081] Step 309 , determine whether the reference distance corresponding to each of the targets is consistent with the predicted distance, if so, execute step 310 , if not, execute steps 311 - 312 .

[0082] Step 310: taking the predicted distance corresponding to each of the targets as the final image object distance fusion result.

[0083] In this step, if the predicted distance corresponding to the target is consistent with the reference distance, that is, when the predicted distance does not need to be changed during the calibration process of the predicted distance, the predicted distance of the target is used as the final image object distance fusion result.

[0084] Step 311: Targets whose reference distances are inconsistent with the predicted distances are regarded as abnormal targets, and the predicted distances corresponding to the targets are regarded as the final image object distance fusion results.

[0085] In this step, if the predicted distance corresponding to the target is inconsistent with the reference distance, that is, when the predicted distance needs to be changed during the calibration process of the predicted distance, the predicted distance of the target is used as the final image object distance fusion result.

[0086] The reason for this setting is that, due to the differences in actual heights or widths between multiple targets, when the predicted distance of the target is inconsistent with the reference distance, if the predicted distance is changed only according to the picture frame area, it is very likely to cause errors in the image object distance fusion result. Therefore, by using the predicted distance as the image object distance fusion result, the accuracy of the image object distance fusion result can be improved.

[0087] Step 312: In the environment image, the frame of the abnormal target is marked with a preset color to prompt the user.

[0088] In this embodiment, when the predicted distance of a target is inconsistent with the reference distance, the target can be regarded as an abnormal target, and the frame of the abnormal target can be marked with a special color in the environmental image to prompt the user to further check the predicted distance of the target.

[0089] The technical solution provided by the embodiment of the present invention obtains the target distance measurement value in real time through the distance sensor, identifies the target existing in the environmental image to obtain the target size information, generates a picture frame corresponding to each target, combines the distance measurement values ​​corresponding to each target to obtain the image name, fills the distance measurement values ​​of each target into the picture frame in sequence, determines the predicted distance of the target according to the distance filling result, sorts the picture frames according to the area of ​​the picture frames, determines the reference distance of the target according to the sorted picture frames and the target predicted distances corresponding to the picture frames, judges whether the reference distance of the target is consistent with the predicted distance, and if not, regards the target as an abnormal target, and regards the predicted distance of each target as the final image object distance fusion result, marks the picture frame of the abnormal target in the environmental image according to a preset color, and uses the technical means to prompt the user, which can reduce the error of the image object distance fusion result and improve the accuracy of the image object distance fusion result.

[0090] Figure 4 A schematic diagram of the structure of an image-object distance fusion device provided in Embodiment 4 of the present invention is shown in FIG. Figure 4 As shown, the device includes: a distance measurement module 410, a target recognition module 420 and a predicted distance determination module 430.

[0091] The distance measurement module 410 is used to obtain the distance measurement value between each target in the monitoring range and the preset position point in real time through the distance sensor;

[0092] The target recognition module 420 is used to collect the environment image corresponding to the monitoring range according to the preset time interval, and recognize each target existing in the environment image to obtain the image recognition result corresponding to each target;

[0093] The predicted distance determination module 430 is used to determine the predicted distance between each target in the environment image and the preset position point according to the distance measurement value between each target and the preset position point and the image recognition result corresponding to each target.

[0094] The technical solution provided by the embodiment of the present invention uses a distance sensor to obtain the distance measurement value between each target in the monitoring range and the preset position point in real time, collects the environmental image corresponding to the monitoring range according to the preset time interval, and identifies each target existing in the environmental image to obtain the image recognition result corresponding to each target, and determines the predicted distance between each target in the environmental image and the preset position point according to the distance measurement value between each target and the preset position point, and the image recognition result corresponding to each target. The technical means can reduce the error of the image object distance fusion result and improve the accuracy of the image object distance fusion result.

[0095] Based on the above embodiment, the distance measurement module 410 includes:

[0096] A coordinate acquisition unit, used to acquire the coordinate information corresponding to each target within the monitoring range in real time through the distance sensor;

[0097] A target arrangement unit, used to arrange the targets in ascending order of horizontal coordinate values;

[0098] The distance list construction unit is used to store the arranged targets, the horizontal coordinate values ​​corresponding to the targets, and the distance measurement values ​​in sequence into the distance list.

[0099] The target recognition module 420 includes:

[0100] A size acquisition unit, used to identify the objects in the environment image and obtain size information corresponding to each object;

[0101] The picture frame generating unit is used to generate a picture frame corresponding to each target in the environment image according to the size information corresponding to each target.

[0102] The predicted distance determination module 430 includes:

[0103] a measurement value combining unit, configured to sequentially obtain the distance measurement values ​​corresponding to the targets in the distance list according to the arrangement order of the targets, and combine the distance measurement values ​​corresponding to the targets to obtain an image name of the environment image;

[0104] A measurement value filling unit, used to fill the distance measurement value corresponding to each target into the picture frame corresponding to each target in sequence according to the image name of the environment image;

[0105] A filling result processing unit, used to determine the predicted distance between each of the targets and a preset position point according to the distance filling result corresponding to each of the picture frames;

[0106] A vertical coordinate value acquisition unit, configured to acquire target vertical coordinate values ​​corresponding to a plurality of picture frames respectively if the target horizontal coordinate values ​​corresponding to the plurality of picture frames are equal;

[0107] A filling result correction unit, used to correct the distance filling result corresponding to each of the picture frames according to the target vertical coordinate value corresponding to each of the picture frames, so as to obtain the predicted distance between each of the targets and the preset position point;

[0108] A picture frame sorting unit, used for sorting the picture frames according to the areas corresponding to the picture frames;

[0109] A reference distance determination unit, configured to determine a reference distance between each target and a preset position point according to the sorted picture frames and the target prediction distance corresponding to each picture frame;

[0110] The distance judgment unit is used to judge whether the reference distance corresponding to each of the targets is consistent with the predicted distance. If so, the predicted distance corresponding to each of the targets is used as the final image object distance fusion result. If not, the target whose reference distance is inconsistent with the predicted distance is regarded as an abnormal target, and the predicted distance corresponding to each of the targets is used as the final image object distance fusion result. In the environmental image, the frame of the abnormal target is marked in a preset color to prompt the user.

[0111] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not fully described in the embodiments of the present invention, please refer to the methods provided by all the above embodiments of the present invention.

[0112] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0113] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0114] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0115] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the image object distance fusion method.

[0116] In some embodiments, the image-object-distance fusion method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the image-object-distance fusion method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image-object-distance fusion method in any other appropriate manner (e.g., by means of firmware).

[0117] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0119] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0121] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0122] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0124] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image object distance fusion method, characterized in that: include: Through the distance sensor, the distance measurement value between each target within the monitoring range and the preset position point is obtained in real time; According to a preset time interval, an environmental image corresponding to the monitoring range is collected, and each target existing in the environmental image is identified to obtain an image recognition result corresponding to each target; Determine the predicted distance between each target and the preset position point in the environment image according to the distance measurement value between each target and the preset position point and the image recognition result corresponding to each target; After obtaining the distance measurement value between each target within the monitoring range and the preset position point in real time, it also includes: The distance sensor is used to obtain the coordinate information of each target in the monitoring range in real time. Arrange the targets in ascending order of horizontal coordinate values; The arranged targets, the horizontal coordinate values ​​corresponding to the targets, and the distance measurement values ​​are sequentially stored in a distance list; Identify each target in the environment image to obtain an image recognition result corresponding to each target, including: Identify the objects in the environment image and obtain size information corresponding to each object; Generating a picture frame corresponding to each target in the environment image according to the size information corresponding to each target; Determining the predicted distance between each target in the environment image and the preset position point according to the distance measurement value between each target and the preset position point and the image recognition result corresponding to each target includes: In the distance list, according to the arrangement order of each target, the distance measurement value corresponding to each target is obtained in turn, and the distance measurement value corresponding to each target is combined to obtain the image name of the environment image; According to the image name of the environment image, the distance measurement value corresponding to each target is filled into the picture frame corresponding to each target in sequence; The predicted distance between each of the targets and a preset position point is determined according to the distance filling results corresponding to each of the picture frames.

2. The method according to claim 1, characterized in that: Determining the predicted distance between each target and a preset position point according to the distance filling result corresponding to each picture frame includes: If there are multiple picture frames whose corresponding target horizontal coordinate values ​​are equal, obtaining the target vertical coordinate values ​​respectively corresponding to the multiple picture frames; According to the vertical coordinate value of the target corresponding to each of the picture frames, the distance filling result corresponding to each of the picture frames is corrected to obtain the predicted distance between each of the targets and the preset position point.

3. The method according to claim 1, characterized in that After determining the predicted distance between each of the targets and the preset position point according to the distance filling results corresponding to each of the picture frames, the method further includes: sorting the picture frames according to the areas corresponding to the picture frames; According to the sorted picture frames and the target predicted distances corresponding to the picture frames, determining the reference distance between each target and a preset position point, wherein the reference distance is the distance after the predicted distance is calibrated; Determining whether the reference distance corresponding to each of the targets is consistent with the predicted distance; If so, the predicted distance corresponding to each of the targets is used as the final image object distance fusion result.

4. The method according to claim 3, characterized in that After determining whether the reference distance corresponding to each of the targets is consistent with the predicted distance, the method further includes: If not, the target whose reference distance is inconsistent with the predicted distance is regarded as an abnormal target, and the predicted distance corresponding to each target is regarded as the final image object distance fusion result; In the environment image, the frame of the abnormal target is marked with a preset color to prompt the user.

5. An image-object distance fusion device, characterized in that: include: The distance measurement module is used to obtain the distance measurement value between each target in the monitoring range and the preset position point in real time through the distance sensor; The target recognition module is used to collect the environment image corresponding to the monitoring range according to the preset time interval, and recognize each target existing in the environment image to obtain the image recognition result corresponding to each target; A predicted distance determination module, used to determine the predicted distance between each target in the environment image and the preset position point according to the distance measurement value between each target and the preset position point and the image recognition result corresponding to each target; The distance measurement module comprises: A coordinate acquisition unit, used to acquire the coordinate information corresponding to each target within the monitoring range in real time through the distance sensor; A target arrangement unit, used to arrange the targets in ascending order of horizontal coordinate values; A distance list construction unit, used to sequentially store the arranged targets, the horizontal coordinate values ​​corresponding to the targets, and the distance measurement values ​​into the distance list; Target recognition module, including: A size acquisition unit, used to identify the objects in the environment image and obtain size information corresponding to each object; A picture frame generating unit, configured to generate a picture frame corresponding to each target in the environment image according to size information corresponding to each target; The prediction distance determination module includes: a measurement value combining unit, configured to sequentially obtain the distance measurement values ​​corresponding to the targets in the distance list according to the arrangement order of the targets, and combine the distance measurement values ​​corresponding to the targets to obtain an image name of the environment image; A measurement value filling unit, used to fill the distance measurement value corresponding to each target into the picture frame corresponding to each target in sequence according to the image name of the environment image; The filling result processing unit is used to determine the predicted distance between each of the targets and the preset position point according to the distance filling results corresponding to each of the picture frames.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image-object distance fusion method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image-object distance fusion method according to any one of claims 1 to 4 when executed.

Images