Object recognition method, object recognition device, equipment, medium and product

Abstract

The embodiment of the invention provides an object recognition method, an object recognition device, equipment, a medium and a product. The object recognition method comprises the following steps: constructing a digital twin space model of a to-be-recognized region; the digital twin space model comprises a plurality of image acquisition equipment models; each image acquisition device model corresponds to a plurality of image acquisition devices pre-configured in the to-be-identified area; determining a virtual identification area corresponding to the to-be-identified area in the digital twin space model and virtual identification sub-areas respectively corresponding to the image acquisition equipment models in the virtual identification area; determining an object recognition area of each image acquisition device based on the virtual recognition sub-area corresponding to each image acquisition device model; and controlling each image acquisition device to perform image acquisition on the respective corresponding object recognition area to obtain a target image, and performing object recognition on the to-be-recognized area based on the target image. According to the embodiment of the invention, object identification can be automatically carried out.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an object recognition method, object recognition device, equipment, medium and product. Background Art

[0002] In real life, especially in production activities, there are usually some areas where pedestrians are not allowed to enter and exit at will, such as high-risk areas or special areas. In order to better protect personal safety and prevent pedestrians from entering and exiting high-risk areas, the existing technology usually conducts video surveillance of high-risk areas and supervises the entry and exit of pedestrians in high-risk areas by manpower. Due to the limited energy of human beings, the negligence of the monitoring background personnel may lead to failure to timely detect people entering high-risk areas.

[0003] In view of the above-mentioned unsafe factors, an object recognition solution is urgently needed to automatically perform object recognition and ensure the accuracy of object recognition. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an object recognition method, an object recognition device, equipment, medium and product to solve the problem that objects cannot be recognized automatically in the related art.

[0005] In order to solve the above technical problems, the embodiments of the present application are implemented as follows: In a first aspect, an embodiment of the present application provides an object recognition method, comprising: Constructing a digital twin space model of the area to be identified; the digital twin space model includes a plurality of image acquisition device models; each of the image acquisition device models corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; Determine a virtual recognition area corresponding to the area to be recognized in the digital twin space model, and virtual recognition sub-areas in the virtual recognition area corresponding to each of the image acquisition device models; Determining the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to the image acquisition device models; Each of the image acquisition devices is controlled to acquire images of the object recognition area corresponding to the respective one to obtain a target image, and based on the target image, object recognition is performed on the area to be recognized.

[0006] In a second aspect, an embodiment of the present application provides an object recognition device, including: A spatial model construction module, used to construct a digital twin spatial model of the area to be identified; the digital twin spatial model includes a plurality of image acquisition device models; each of the image acquisition device models corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; A virtual area determination module, used to determine a virtual identification area corresponding to the area to be identified in the digital twin space model, and virtual identification sub-areas in the virtual identification area corresponding to each of the image acquisition device models; A recognition area determination module, used to determine the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to each image acquisition device model; The object recognition module is used to control each of the image acquisition devices to perform image acquisition on the corresponding object recognition area to obtain a target image, and based on the target image, perform object recognition on the area to be recognized.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the object recognition method described in the first aspect above are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the object recognition method described in the first aspect above are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the steps of the object recognition method described in the first aspect above.

[0010] It can be seen from the technical solution provided by the above embodiments of the present application that, in the embodiments of the present application, first, a digital twin space model of the area to be identified is constructed; the digital twin space model includes multiple image acquisition device models; each image acquisition device model corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; then, the virtual identification area corresponding to the area to be identified in the digital twin space model and the virtual identification sub-areas corresponding to each image acquisition device model in the virtual identification area are determined; then, based on the virtual identification sub-areas corresponding to each image acquisition device model, the object identification area of ​​each image acquisition device is determined; finally, each image acquisition device is controlled to perform image acquisition on its corresponding object identification area to obtain a target image, and based on the target image, object identification is performed on the area to be identified. It can be seen that through the embodiments of the present application, the digital twin space model can be synchronized with the area to be identified in real time, and the object recognition area of ​​each image acquisition device is determined by the virtual recognition sub-areas corresponding to each image acquisition device model in the digital twin space model, and then each image acquisition device is controlled to perform image acquisition on its corresponding object recognition area to obtain the target image, so as to perform object recognition on the area to be identified based on the target image, which enables the image acquisition device in the area to be identified to capture the changes in the image of the object recognition area in a timely and accurate manner, thereby improving the accuracy and efficiency of object recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0012] Figure 1 A flowchart of an object recognition method provided in an embodiment of the present application; Figure 2a A schematic diagram of an image of a portion of a well site provided in an embodiment of the present application; Figure 2b A schematic diagram of a spliced ​​image of a part of a well site provided in an embodiment of the present application; Figure 2c A schematic diagram of an overall spliced ​​image of a well site provided in an embodiment of the present application; Figure 3 An exemplary flow chart of an object recognition method provided in an embodiment of the present application; Figure 4 A schematic diagram of the module composition of an object recognition device provided in an embodiment of the present application; Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] Embodiments of the present application provide an object recognition method, an object recognition device, a device, a medium, and a product.

[0014] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0015] Before introducing the object recognition method, object recognition device, equipment, medium and product provided in the embodiment of the present application, the application scenario provided in the embodiment of the present application is first described. The object recognition method, object recognition device, equipment, medium and product provided in the embodiment of the present application can be applied to the scene of personnel recognition in the area to be recognized. In a specific application scenario, the area to be recognized may be an area with high-risk or high-pressure operations such as a well site. For example, during the fracturing operation at the well site, there is a high-pressure operation area, and workers are not allowed to enter and exit the high-pressure area. If a person enters the high-pressure area during the fracturing construction to collect images of the hydraulic end operation, if the high-pressure area pipeline bursts at this time, it will cause a safety risk. In this scenario or similar scenarios, it is necessary to identify personnel in high-risk or high-pressure areas to avoid safety risks.

[0016] The object recognition method, object recognition device, equipment, medium and product provided in the embodiments of the present application are clearly and completely described below in combination with specific application scenarios and the drawings in the embodiments of the present application.

[0017] Figure 1 The present invention provides a flowchart of an object recognition method, which can be executed by a server, which can be an independent server, a server cluster composed of multiple servers, or a cloud server for cloud computing processing. Figure 1 As shown, the object recognition method may specifically include the following steps: S102: Construct a digital twin space model of the area to be identified; the digital twin space model includes multiple image acquisition device models; each image acquisition device model corresponds to a plurality of image acquisition devices pre-configured in the area to be identified.

[0018] In an embodiment of the present application, a digital twin space model of the area to be identified is constructed, and the digital twin space model includes multiple image acquisition device models corresponding to multiple image acquisition devices in the area to be identified. Among them, multiple image acquisition devices are pre-installed and deployed in the area to be identified, and the area to be identified may be an area where object identification is required. The image acquisition device in the area to be identified may be a camera with an image acquisition function.

[0019] In one implementation, the area to be identified may be a high-pressure area and an area close to the high-pressure area in the well site, the area close to the high-pressure area being an area within a preset range of distance from the high-pressure area, and the image acquisition device may be a camera installed in the high-pressure area and an area close to the high-pressure area of ​​the well site, for capturing images or videos of the high-pressure operation area and the area close to the high-pressure area of ​​the well site.

[0020] In another implementation, a digital twin space model of the well site as a whole can also be constructed, and the area to be identified that needs to be identified is determined in the constructed digital twin space model of the well site as a whole. The image acquisition equipment in the well site can include an overall image acquisition device and a local image acquisition device for the well site, and the overall image acquisition device and the local image acquisition device can be cameras. The overall image acquisition device of the well site can be installed on a high searchlight of the well site, or installed above a water tank of the well site, for long-distance image acquisition of the entire well site. The local image acquisition device is used for close-range image acquisition of the high-pressure operation area of ​​the well site. The local image acquisition device can be installed on the monitoring tooling arranged on both sides of the high and low pressure manifold skid, or it can be installed on the upper bracket of the fracturing skid close to the high and low pressure manifold skid, and can also be installed in other locations to achieve full coverage image acquisition of the fracturing high pressure area.

[0021] The relative position information and related parameter information of each image acquisition device model in the digital twin space model correspond one-to-one to the position and parameter information of each image acquisition device in the well site.

[0022] In one embodiment, constructing a digital twin spatial model of the area to be identified includes: Acquire a two-dimensional image and a three-dimensional image of the area to be identified; Construct a preset device model of each preset device in the area to be identified in the digital twin space model; Based on the two-dimensional image and the three-dimensional image, as well as the identification information of each preset device, determine the location information of each preset device model in the digital twin space model; Based on two-dimensional images and three-dimensional images, as well as the location information of each preset equipment model in the digital twin space model, a digital twin space model of the area to be identified is constructed.

[0023] In an embodiment of the present application, to construct a digital twin space of the area to be identified, first, it is necessary to obtain a two-dimensional image and a three-dimensional image of the area to be identified; then, a preset device model of each device pre-configured in the area to be identified in the digital twin space model is constructed; then, based on the two-dimensional image and the three-dimensional image of the area to be identified and the identification information of each preset device, the position information of each preset device model in the digital twin space model is determined; finally, based on the two-dimensional image and the three-dimensional image of the area to be identified, and the position information of each preset device model in the digital twin space model, the digital twin space model of the area to be identified is constructed.

[0024] In one implementation, the two-dimensional image of the area to be identified can be a color overhead image of the area to be identified, and the three-dimensional image can be a three-dimensional point cloud image of the area to be identified. Taking the well site as an example, the color overhead image and the three-dimensional point cloud image of the well site can be obtained by the positioning device and the ranging sensor suspended by the drone to collect images of the well site. Among them, the positioning device can be a gyroscope, which is used to determine the real-time posture (position and attitude) of the ranging sensor. The ranging sensor can be a binocular camera, a depth camera, etc. The ranging sensor can collect the color overhead image of the well site on the one hand, and can perform spatial ranging to collect the three-dimensional point cloud image of the well site on the other hand.

[0025] When determining the real-time position and attitude of the ranging sensor, the RTK (Real Time Kinematic) positioning technology can be used to determine the position of the UAV. The RTK system includes a rover and a base station. The base station can be fixed on the top of the well site equipment, such as the top of a water tank or the top of a fracturing truck. The rover can be installed and fixed on the UAV as part of the positioning device of the UAV, and the position of the UAV is calculated based on the RTK algorithm. A fixed inertial measurement unit (Imu) can also be installed on the UAV as part of the positioning device of the UAV. The Imu is used to determine the attitude of the UAV. The ranging sensor is installed and fixed on the UAV, and the attitude of the UAV is the attitude of the ranging sensor.

[0026] The acquisition process of the color top view of the well site is as follows: the UAV is controlled to use the ranging sensor to collect the color top view of the well site in real time during the flight process. According to the relative position changes of the positioning device on the UAV at different times, the transformation matrix between the multiple color top views collected by the ranging sensor can be calculated. Using this transformation matrix, the color top views taken by the UAV ranging sensor at different positions are image stitched to obtain a two-dimensional color top view of the well site as a whole.

[0027] For example, Figure 2aThis is a schematic diagram of an image of a part of a well site provided in an embodiment of the present application. The two images are obtained by capturing images of two areas in the well site by a ranging sensor. The two images are spliced ​​together by a transformation matrix to obtain a spliced ​​image. The spliced ​​image is shown in FIG. Figure 2b On this basis, the multiple images collected by the ranging sensor can be spliced ​​through the transformation matrix to obtain the overall spliced ​​image of the well site. Figure 2c This is a schematic diagram of the stitched image of the entire well site.

[0028] The acquisition process of the three-dimensional point cloud map of the well site is as follows: the UAV is controlled to use the ranging sensor to collect the three-dimensional point cloud map of the well site in real time during flight. According to the relative position changes of the positioning device on the UAV at different times, the spatial transformation matrix between different point cloud maps can be calculated. The point cloud maps taken by the UAV at different positions are stitched using the spatial transformation matrix to obtain the overall three-dimensional point cloud image of the well site.

[0029] After collecting the two-dimensional color overhead view image and three-dimensional point cloud image of the well site, a stitching transformation matrix between the well site color overhead view image and the overall three-dimensional point cloud image of the well site can be established based on the transformation matrix between the positioning device and the ranging sensor on the UAV, the plane stitching transformation matrix of each color overhead view image taken by the UAV relative to the well site color overhead view image, and the stereo stitching transformation matrix of each point cloud image relative to the overall three-dimensional point cloud image of the well site.

[0030] The determination process of the above-mentioned splicing matrices can refer to the relevant technology, and this application will not go into details here.

[0031] Next, it is necessary to determine the scale of the digital twin space model of the well site relative to the well site, and the scale is, for example, 1:1. The origin and direction of the digital twin space model are selected, and the conversion relationship between the digital twin space model and the three-dimensional point cloud space of the well site is established. The three-dimensional point cloud space of the well site can be obtained by sampling the well site and each device in the well site through three-dimensional scanning technology (such as laser scanning, structured light scanning, etc.), which contains the precise position and shape information of the well site and each device in the well site in the three-dimensional space. The three-dimensional point cloud space of the well site corresponds to the three-dimensional point cloud image of the well site, and can both represent the precise position and shape information of the well site and each device in the well site in the three-dimensional space. Subsequently, the equipment model of each device in the well site in the digital twin space model is established. The equipment in the well site is, for example, a fracturing skid, an instrument skid, a high- and low-pressure manifold skid, and the like. Accordingly, it is necessary to construct an equipment model corresponding to the fracturing skid, the instrument skid, the high- and low-pressure manifold skid, and the like in the well site in the digital twin space model. The location information of each equipment model in the digital twin space model needs to correspond one-to-one with the location information of each equipment in the well site. Therefore, it is also necessary to locate each equipment model in the digital twin space model.

[0032] There are usually multiple devices of the same model installed in the well site, such as multiple fracturing trucks of the same model. Each fracturing truck is located in a different position in the well site. In this case, the devices can be distinguished according to the identification information of each device. For example, a different device label can be set for each device. The device label can be a device number placed on the device by pasting or magnetic attraction, such as A001, B002, etc., and each device number is different. It can also be a barcode with a unique identification placed on the device by pasting or magnetic attraction. The barcode can be, for example, a one-dimensional code or a two-dimensional code. The device number or serial number (SN) of the device can be read by scanning the barcode. The device label can also be numbers, letters or symbols, and the numbers, letters or symbols on each device are different. The device label of the device can be placed in a fixed position on the device.

[0033] When positioning each equipment model in the digital twin space model, first, the well site color image top view image can be input into the pre-trained equipment recognition model to obtain the equipment recognition result. The equipment recognition model can be obtained by pre-training the target recognition algorithm model. The training samples of the equipment recognition model include the well site color image top view sample image and each well site equipment in the well site color image top view sample image. The target recognition algorithm model is iteratively trained using the training samples to obtain the equipment recognition model. The target recognition algorithm model is, for example, a target recognition algorithm model such as YOLO. The equipment to be identified includes the fracturing skid, instrument skid, high and low pressure manifold skid, sand transport skid and other operating equipment in the well site, and may also include the well site overall image acquisition equipment and local image acquisition equipment in the well site. Then, based on the equipment label of each equipment and the splicing conversion matrix between the well site color image top view image and the well site overall three-dimensional point cloud image, the location information of each equipment model is determined in the digital twin space model. The location information of the equipment model includes position and direction.

[0034] Specifically, the location information of each device model can be determined in the following two ways. One way is: based on the device label of each device and the splicing conversion matrix between the top view image of the well site color image and the overall three-dimensional point cloud image of the well site, the approximate position of each identified device in the overall three-dimensional point cloud space of the well site is determined, and this position is used as the initial positioning position of the device. Then, based on the initial positioning position of each device, the device model corresponding to each device is placed at the initial positioning position in the overall three-dimensional point cloud space of the well site, and then further positioning fine-tuning is performed to obtain the precise positioning of each device model in the digital twin space model. The process of accurately positioning each device model in the digital twin space model is as follows: (1) According to the initial positioning position of each device model in the three-dimensional point cloud space of the well site, the point cloud data of other positions in the three-dimensional point cloud space of the well site except the initial positioning position is temporarily removed, and only the point cloud data corresponding to the device model is calculated and analyzed to prevent subsequent analysis and calculation interference. (2) The device model is three-dimensionally expanded, for example, the size of the device model can be increased, and the expansion amount can be, for example, 2 cm, and then the number of point clouds in the expanded device model is counted. (3) The equipment model is rotated around the center of the model at a specified rotation angle each time, such as 5 degrees. The number of point clouds in the equipment model after rotation is counted each time, and the direction with the largest number of point clouds in the equipment model is used as the accurate positioning direction of the equipment model. (4) The equipment model is moved forward and backward based on the accurate positioning direction determined above, each time according to the specified value, such as 5 centimeters each time, and the position with the largest number of point clouds in the equipment model is used as the accurate positioning position of the equipment model. (5) The above-mentioned three-dimensional point cloud space of the well site is converted into a digital twin space.

[0035] All equipment models in the digital twin space model are accurately positioned in the above manner.

[0036] Another way is: based on the splicing conversion matrix between the well site color image and the overall three-dimensional point cloud image of the well site, determine the approximate spatial position of each device in the well site relative to the current drone, and plan the drone's path based on the overall three-dimensional point cloud image of the well site. The drone flies to a certain device, bypasses the device and identifies the device tag of the device. According to the position of the device tag on the device and the orientation of the drone relative to the device tag, the accurate orientation of the device is determined. In this way, the location information of each device is determined, and then, based on the location information of each device in the well site, all device models are accurately positioned in the digital twin space model.

[0037] In addition, the relevant parameters of the image acquisition device model in the digital twin space model need to be adjusted. The relevant parameters of the well site overall image acquisition device and local image acquisition device in the well site cannot be measured by the ranging sensor on the drone. For example, the horizontal angle, pitch angle, focal length, horizontal field of view, vertical field of view and other data of the pan-tilt rotation of the ball camera in the well site overall image acquisition device cannot be measured by the ranging sensor. The relevant parameters of the image acquisition device and the pan-tilt can be obtained by communicating with the image acquisition device in the well site. According to the relevant parameters of the image acquisition device and the pan-tilt in the well site, the parameters of the image acquisition device model in the digital twin space model of the well site can be adjusted.

[0038] S104: Determine the virtual recognition area corresponding to the area to be recognized in the digital twin space model, and the virtual recognition sub-areas in the virtual recognition area corresponding to each image acquisition device model.

[0039] In an embodiment of the present application, after constructing the digital twin space model of the area to be identified, it is necessary to determine the virtual identification area corresponding to the area to be identified in the digital twin space model, as well as the virtual identification sub-areas in the virtual identification area that correspond one-to-one to each image acquisition device model.

[0040] In one implementation, the area to be identified is, for example, the high-pressure area between two rows of fracturing trucks in the well site, and the area surrounded by all fracturing trucks and wellheads. When determining the virtual identification area corresponding to the area to be identified in the digital twin space model, the area between the two rows of fracturing trucks in the digital twin space model can be marked (can be marked with a specific color) as the high-pressure area, and the area surrounded by all fracturing trucks and wellheads in the digital twin space model can be marked as close to the high-pressure area.

[0041] In one embodiment, determining the virtual recognition sub-areas in the virtual recognition area corresponding to each image acquisition device model includes: For each image acquisition device model, determining a plurality of spatial positions whose distances from the image acquisition device model are within a preset distance range; The regions corresponding to the multiple spatial positions are determined as virtual identification sub-regions corresponding to the image acquisition device model.

[0042] In one implementation, when determining the virtual identification sub-areas corresponding to each image acquisition device model in the virtual identification area, first, for each image acquisition device model, multiple spatial positions whose distances to the image acquisition device model are within a preset distance range are determined, and then the area composed of the multiple spatial positions is determined as the virtual identification sub-area corresponding to the image acquisition device model.

[0043] In another implementation, the virtual recognition area can be divided according to the number of image acquisition device models to obtain multiple virtual recognition sub-areas, and each virtual recognition sub-area can be marked (may be marked with a specific color, texture, etc.) to distinguish each virtual recognition sub-area. Then, for each image acquisition device model, the virtual recognition sub-area closest to the image acquisition device model is determined, and the virtual recognition sub-area is determined as the virtual recognition sub-area corresponding to the image acquisition device model.

[0044] In one implementation, the image acquisition device model can also be used to perform "image acquisition" on the corresponding virtual recognition sub-area. Determine whether the entire virtual recognition sub-area is within the "image acquisition" range of the image acquisition device model. If the entire virtual recognition sub-area is not within the "image acquisition" range of the image acquisition device model, adjust the angle of the pan / tilt on the image acquisition device model until the entire virtual recognition sub-area is within the "image acquisition" range of the image acquisition device model. After that, the pan / tilt rotation angle and the virtual recognition sub-area corresponding to each image acquisition device model can be recorded to determine and draw the "acquisition outline" of the virtual recognition sub-area by the image acquisition device model. In this way, the determination result of the virtual recognition sub-area corresponding to each image acquisition device model can be made more accurate. It can be understood that the image acquisition device model, that is, the virtual camera, is used in the digital twin space model to simulate the real camera in the area to be identified. The "image acquisition" process of the virtual camera is actually realized by computer graphics rendering technology, which is the process of rendering the three-dimensional model, lighting, texture and other elements in the virtual scene into two-dimensional images. These images can be static screenshots or dynamic video sequences.

[0045] S106: Determine the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to the models of each image acquisition device.

[0046] In an embodiment of the present application, after determining the virtual identification sub-areas corresponding to each image acquisition device model, the object identification area of ​​each image acquisition device in the area to be identified is determined based on the virtual identification sub-areas corresponding to each image acquisition device model in the digital twin space model.

[0047] In one embodiment, determining the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to each image acquisition device model includes: Determine parameter information of each image acquisition device model based on the virtual identification sub-areas respectively corresponding to each image acquisition device model; Based on the parameter information of each image acquisition device model, adjusting the parameter information of each image acquisition device in the area to be identified; Based on the adjusted parameter information of each image acquisition device, an object recognition area of ​​each image acquisition device is determined.

[0048] In one embodiment, the parameter information of the image acquisition device model includes: the focal length of the image acquisition device model and the pan / tilt angle of the image acquisition device model; Based on the parameter information of each image acquisition device model, the parameter information of each image acquisition device in the area to be identified is adjusted, including: Based on the focal length and pan / tilt angle of each image acquisition device model, the focal length and pan / tilt angle of each image acquisition device in the area to be identified are adjusted.

[0049] In one implementation, when determining the object recognition area of ​​each image acquisition device based on the virtual recognition sub-area corresponding to each image acquisition device model, first, the focal length and pan-tilt angle of each image acquisition device model can be determined according to the virtual recognition sub-area corresponding to each image acquisition device model; then, according to the focal length and pan-tilt angle of each image acquisition device model, the parameter information of each image acquisition device in the area to be recognized is adjusted, and the focal length and pan-tilt angle of each image acquisition device after adjustment are the same as the focal length and pan-tilt angle of the corresponding image acquisition device model. Finally, according to the adjusted focal length and pan-tilt angle of each image acquisition device, the object recognition area of ​​each image acquisition device in the area to be recognized is determined.

[0050] In another implementation method, the focal length and pan / tilt angle of the image acquisition device in the area to be identified can also be adjusted according to the "acquisition outline" of the virtual identification sub-area by the drawn image acquisition device model, until the "acquisition outline" of the image acquisition device in the area to be identified for the corresponding object identification area is the same as the "acquisition outline" of the image acquisition device model for the virtual identification sub-area, and the object identification area of ​​each image acquisition device in the area to be identified is determined.

[0051] S108: Control each image acquisition device to acquire images of the corresponding object recognition area to obtain a target image, and perform object recognition on the area to be recognized based on the target image.

[0052] In the embodiment of the present application, after determining the object recognition area of ​​each image acquisition device, each image acquisition device in the area to be recognized can be controlled to perform image acquisition on the corresponding object recognition area to obtain a target image, which can include multiple target images acquired by multiple image acquisition devices. Then, based on the multiple target images acquired by the multiple image acquisition devices, object recognition is performed on the area to be recognized.

[0053] In one embodiment, based on the target image, object recognition is performed on the area to be recognized, including: Determining whether the clarity of the target image is greater than or equal to a preset clarity threshold; When it is determined that the clarity of the target image is greater than or equal to a preset clarity threshold, performing object recognition on the area to be recognized based on the target image by using an object recognition model; When it is determined that the clarity of the target image is less than a preset clarity threshold, the pre-deployed radar equipment is used to perform object recognition on the area to be recognized.

[0054] In the embodiment of the present application, when performing object recognition in the area to be recognized, it can be determined whether the clarity of the target image captured by the image acquisition device is greater than or equal to the preset clarity threshold. If it is determined that the clarity of the target image captured by the image acquisition device is greater than or equal to the preset clarity threshold, the object recognition in the area to be recognized is performed based on the target image using a pre-trained object recognition model. If it is determined that the clarity of the target image is less than the preset clarity threshold, the radar device pre-deployed in the area to be recognized is used to perform object recognition in the area to be recognized.

[0055] The object recognition model may be a model capable of object recognition obtained by iteratively training a pre-built neural network using training samples in advance. The training samples may include multiple sample images, and the labels are pedestrians in each sample image.

[0056] It can be understood that when the clarity of the target image captured by the camera is greater than or equal to the preset clarity threshold, the object recognition model can accurately identify the target object from the target image with sufficiently high clarity. When the focal length and pan / tilt angle of the camera are fixed, the clarity of the image is easily affected by light, etc. Compared with the camera, the radar is less affected by light when performing object recognition. Therefore, when the clarity of the target image captured by the image acquisition device is less than the preset clarity threshold, the radar device can be used to perform object recognition on the area to be recognized.

[0057] In one implementation, the radar device may be a millimeter-wave radar or a laser radar for 360-degree detection in the horizontal direction. The millimeter-wave radar may be combined to achieve 360-degree detection. For example, if a millimeter-wave radar has a horizontal detection range of 180 degrees, two millimeter-wave radars may be combined to achieve 360-degree detection.

[0058] In one implementation, a light detection device (such as a illuminance meter) may also be used to detect the light brightness of the area to be identified. When the light brightness is less than a preset brightness threshold, the light condition is considered to be poor, and the radar device is activated to perform object identification.

[0059] In one embodiment, object recognition is performed on the area to be recognized based on the target image by using the object recognition model, including: Inputting the target image into the object recognition model to obtain a recognition result of the target image; If the recognition result is that the target object is recognized, an alarm is issued for the area to be recognized based on a predetermined alarm strategy.

[0060] In the embodiment of the present application, when the object recognition model is used to perform object recognition on the area to be recognized based on the target image, multiple target images captured by each image acquisition device in the area to be recognized can be sequentially input into the object recognition model to obtain the recognition results of each target image. If the recognition result output by the object recognition model is that the target object is recognized in the target image, then, according to a predetermined alarm strategy, an alarm is issued for the area to be recognized.

[0061] In one implementation, the area to be identified is the high-pressure area and the area close to the high-pressure area in the well site, and the area close to the high-pressure area is the area whose distance from the high-pressure area is within a preset range. The images taken by each camera in the high-pressure area and the area close to the high-pressure area are input into the object recognition model to obtain the recognition results of each image. If the recognition result is that a pedestrian is recognized in the image, it means that a pedestrian has entered the high-pressure area or is approaching the high-pressure area. In this case, an alarm can be issued in the high-pressure area and the area close to the high-pressure area according to a predetermined alarm strategy.

[0062] In one implementation, the alarm strategy is: when it is detected that someone breaks into the area near the high-voltage area, a non-emergency alarm is activated, and the well site issues a voice alarm reminder, such as "Dangerous area ahead, please leave", to remind people to leave the area near the high-voltage area, and at the same time, the camera in the high-voltage area is turned on. When an emergency alarm is detected that someone breaks into the high-voltage area, an emergency alarm is activated, and the well site issues a voice alarm reminder, such as "Danger, danger, please leave quickly", to remind people to leave the high-voltage area quickly. After review by the background personnel, the corresponding emergency handling of the danger is carried out in conjunction, such as directing people in the high-voltage area to safely exit the high-voltage area or suspending the operation of certain equipment.

[0063] It is understandable that performing personnel intrusion detection on the area near the high-voltage area can avoid the possibility of personnel intruding into the high-voltage area in advance. In another implementation, personnel intrusion detection can also be performed only on the high-voltage area.

[0064] It can be seen from the technical solution provided by the above embodiments of the present application that, in the embodiments of the present application, first, a digital twin space model of the area to be identified is constructed; the digital twin space model includes multiple image acquisition device models; each image acquisition device model corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; then, the virtual identification area corresponding to the area to be identified in the digital twin space model and the virtual identification sub-areas corresponding to each image acquisition device model in the virtual identification area are determined; then, based on the virtual identification sub-areas corresponding to each image acquisition device model, the object identification area of ​​each image acquisition device is determined; finally, each image acquisition device is controlled to perform image acquisition on its corresponding object identification area to obtain a target image, and based on the target image, object identification is performed on the area to be identified. It can be seen that through the embodiments of the present application, the digital twin space model can be synchronized with the area to be identified in real time, and the object recognition area of ​​each image acquisition device is determined by the virtual recognition sub-areas corresponding to each image acquisition device model in the digital twin space model, and then each image acquisition device is controlled to perform image acquisition on its corresponding object recognition area to obtain the target image, so as to perform object recognition on the area to be identified based on the target image, which enables the image acquisition device in the area to be identified to capture the changes in the image of the object recognition area in a timely and accurate manner, thereby improving the accuracy and efficiency of object recognition.

[0065] Corresponding to the object recognition method provided in the above embodiment, the embodiment of the present application also provides an object recognition system, which includes an image acquisition unit, a data transmission unit, a command center unit and an object recognition unit. Among them, the image acquisition unit includes various image acquisition devices in the area to be recognized, which are used to perform image acquisition in the area to be recognized and obtain a target image of the area to be recognized. The data transmission unit is used to transmit the images, video data, etc. collected by each image acquisition device to the command center unit for data analysis. The command center unit includes various preset devices in the area to be recognized, receives the data transmitted by the data transmission unit and performs corresponding data analysis and processing. The object recognition unit is used to perform object recognition on the area to be recognized based on the target image.

[0066] In one implementation, the image acquisition device may include an overall image acquisition device for the well site set up on the well site, which is installed at a relatively high position and can be used for long-distance image acquisition of the entire well site. The overall image acquisition device for the well site can be a camera installed on a high searchlight or a camera installed above a water tank. The image acquisition device may also include a local image acquisition device for close-range image acquisition of the high-pressure area in the well site. The local image acquisition device may be installed on the monitoring tooling on both sides of the high- and low-pressure manifold skid, may be installed on the upper bracket of the fracturing skid close to the high- and low-pressure manifold skid, or may be installed at other locations to achieve full coverage image acquisition of the fracturing high-pressure area. The image acquisition device may also include a three-dimensional spatial ranging sensor, which may be a positioning device and a ranging sensor suspended on a drone, and the ranging sensor may be a binocular camera, a depth camera, etc. The positioning device is used to determine the real-time posture of the ranging sensor, and the ranging sensor is used to capture a color top view of the well site and measure the distance of the well site.

[0067] The data transmission unit is used to transmit the images, videos and other data collected by the overall image acquisition equipment, local image acquisition equipment and three-dimensional space ranging sensors of the well site to the command center unit for subsequent data analysis. The data transmission unit can be a wireless bridge, which can be installed and deployed around each camera to transmit the images or videos and other data taken by each camera. The images and other data taken by the overall image acquisition equipment and local image acquisition equipment of the well site can be wirelessly transmitted to the command center unit through the wireless bridge network, or can be wired to the command center unit through a network cable connection, or can be transmitted wirelessly and wired simultaneously. The three-dimensional space ranging sensor moves dynamically relative to the well site, so wireless transmission can be used for data transmission.

[0068] The command center unit, including the instrument skid at the well site and all the software and hardware equipment in the instrument skid, receives the images, videos and other data transmitted by the data transmission unit and performs corresponding data analysis and processing, including the construction of object recognition models, the splicing of the three-dimensional point cloud map of the well site, and the construction of the digital twin space model. In addition, after analyzing and processing the data, the command center unit can also send control instructions to the corresponding equipment through the wireless bridge. For example, the command center unit sends control instructions to the overall image acquisition equipment of the well site to control the focal length and angle of view of the overall image acquisition equipment of the well site.

[0069] The object recognition unit includes a model recognition module and a radar recognition module, wherein the hardware in the model recognition module includes a workstation, the model recognition module is used to build an object recognition model and perform model training, and the object recognition model is deployed to the corresponding workstation. The object recognition model can be used to perform pedestrian recognition on images or videos taken by the overall image acquisition device and the local image acquisition device of the well site. The radar recognition module includes a radar device and a light detection device, wherein the light detection device, such as an illuminance meter, is used to detect light intensity, and the radar device can be installed and deployed in the high-voltage area of ​​the well site to identify the intrusion of personnel in the high-voltage area.

[0070] When using radar equipment for pedestrian recognition, the proportional relationship between the actual space and the radar equipment interface image can be determined through testing. For example, the number of pixels in the radar equipment interface image corresponding to each 1-meter interval in the actual space can be determined. In this way, the information in the radar equipment interface image can be converted into distance or position information in the actual space. The high-voltage area in the digital twin space model is converted to the corresponding area in the radar equipment interface image. This process can be simplified as converting the top view (two-dimensional image) of the high-voltage area into the corresponding area in the radar equipment interface image (two-dimensional image). Specifically, the position of the radar equipment displayed in the radar equipment interface image is used as the origin to determine the direction and proportion of the top view of the high-voltage area mapped to the corresponding area of ​​the radar equipment interface image. The direction can be determined by the staff through the direction arrow selection on the radar equipment interface image, and the proportion can be determined by the staff manually selecting the two distance lengths in the radar equipment interface image and the high-pressure area top view to make them equal and corresponding. Specifically, on the radar equipment interface image, the staff selects the image distance length from the radar equipment to a certain fixed equipment, such as the tail of a certain fracturing skid, on the radar equipment interface; in the high-pressure area top view, select the image distance length from the radar equipment to the above-mentioned corresponding fixed equipment, and then calculates the proportional relationship between the radar equipment interface image and the high-pressure area top view based on the image distance lengths in the two images.

[0071] The radar device realizes pedestrian intrusion recognition in the high-voltage area by extracting point cloud data, filtering static points, clustering point clouds, etc. The specific implementation process of the above-mentioned steps of extracting point cloud data, filtering static points, clustering point clouds, etc. can refer to the relevant technology, and this application will not repeat it here.

[0072] Figure 3 An exemplary flow chart of the object recognition method provided in the embodiment of the present application is as follows: Figure 3 As shown, in one example, the object recognition method specifically includes the following steps: S301: deploy multiple image acquisition devices in the well site; the image acquisition devices are cameras.

[0073] S302: Collecting a color top view image and a three-dimensional point cloud image of the well site.

[0074] Specifically, a positioning device and a ranging sensor are hung on the UAV, and the color overhead view image and three-dimensional point cloud image collected by the UAV at various positions in the well site are obtained by using the positioning device and the ranging sensor hung on the UAV, and each color overhead view image and three-dimensional point cloud image are spliced ​​to obtain the color overhead view image and three-dimensional point cloud image of the entire well site.

[0075] S303: Set equipment labels for each device in the well site, build equipment models for each device in the well site in the digital twin space model, and build the digital twin space model of the well site.

[0076] Specifically, the equipment in the well site includes operating equipment such as fracturing skids, instrument skids, high and low pressure manifold skids, and various cameras in the well site. A corresponding barcode can be set for each device in the well site as an equipment label for uniquely identifying the device. The equipment model includes the equipment model corresponding to each operating equipment in the well site and the virtual camera corresponding to each camera.

[0077] S304: Determine the location information of each equipment model in the well site digital twin space model.

[0078] Specifically, the position and orientation of each equipment model in the digital twin space model can be determined based on the overall color bird's-eye view image and three-dimensional point cloud image of the well site, as well as the equipment labels of each equipment in the well site.

[0079] S305: Determine a virtual identification area of ​​the to-be-identified area of ​​the well site in the digital twin space model, and determine virtual identification sub-areas corresponding to each virtual camera.

[0080] Specifically, the area to be identified in the well site can be the high-pressure area between two rows of fracturing trucks in the well site and the area close to the high-pressure area surrounded by all fracturing trucks and wellheads. Accordingly, the high-pressure area and the area close to the high-pressure area are determined as virtual identification areas in the digital twin space model. For each virtual camera, multiple spatial positions whose distances to the virtual camera are within a preset distance range are determined, and the area composed of multiple spatial positions is determined as the virtual identification sub-area corresponding to the virtual camera.

[0081] S306: Adjust the focal length and pan / tilt angle of each virtual camera in the digital twin space model.

[0082] Specifically, the focal length and pan / tilt angle of each virtual camera are adjusted so that the virtual camera can "shoot" the entire corresponding virtual identification sub-area. It can be understood that the "shooting" process of the virtual camera is actually achieved through computer graphics rendering technology, which is the process of rendering the three-dimensional model, lighting, texture and other elements in the virtual scene into a two-dimensional image. After adjusting the focal length and pan / tilt angle of each virtual camera, the "capture outline" of each virtual camera for the corresponding virtual sub-area can also be determined and drawn.

[0083] S307: According to the focal length and pan / tilt angle of each virtual camera in the digital twin space model, the focal length and pan / tilt angle of each camera in the well site are adjusted, and the recognition area of ​​each camera in the well site is determined.

[0084] Specifically, the focal length and pan / tilt angle of each camera in the well site after adjustment are the same as the focal length and pan / tilt angle of each virtual camera in the digital twin space model.

[0085] S308: Control each camera in the well site to collect images of the corresponding recognition area to obtain a target image.

[0086] Specifically, the target image includes a plurality of images captured by each camera of the corresponding recognition area.

[0087] S309: Determine whether the clarity of the target image is greater than or equal to a preset clarity threshold.

[0088] Specifically, if the definition of the target image is greater than or equal to a preset definition threshold, S310 is executed; if the definition of the target image is less than the preset definition threshold, S311 is executed.

[0089] S310: Performing object recognition on the area to be recognized by using an object recognition model.

[0090] S311: Perform object recognition in the area to be recognized by using a radar device.

[0091] The specific implementation process and implementation method of this embodiment can refer to the above Figure 1 Various embodiments and implementations of an object recognition method are provided.

[0092] Corresponding to the object recognition method provided in the above embodiment, based on the same technical concept, the embodiment of the present application also provides an object recognition device, Figure 4 A schematic diagram of the module composition of an object recognition device provided in an embodiment of the present application, wherein the user terminal is used to execute Figure 1 Object recognition methods described, such as Figure 4 As shown, the object recognition device includes: A spatial model construction module 401 is used to construct a digital twin spatial model of the area to be identified; the digital twin spatial model includes a plurality of image acquisition device models; each of the image acquisition device models corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; A virtual area determination module 402 is used to determine a virtual identification area corresponding to the area to be identified in the digital twin space model, and virtual identification sub-areas in the virtual identification area corresponding to each of the image acquisition device models; The recognition area determination module 403 is used to determine the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to each image acquisition device model; The object recognition module 404 is used to control each of the image acquisition devices to perform image acquisition on the corresponding object recognition area to obtain a target image, and perform object recognition on the area to be recognized based on the target image.

[0093] In a possible implementation, the space model building module 401 is specifically used to obtain a two-dimensional image and a three-dimensional image of the area to be identified; Constructing a preset device model of each preset device in the to-be-identified area in the digital twin space model; Based on the two-dimensional image and the three-dimensional image and the identification information of each of the preset devices, determine the position information of each of the preset device models in the digital twin space model; Based on the two-dimensional image and the three-dimensional image of the target area, and the position information of each of the preset equipment models in the digital twin space model, a digital twin space model of the area to be identified is constructed.

[0094] In a possible implementation, the virtual area determination module 402 is specifically configured to determine, for each of the image acquisition device models, a plurality of spatial positions whose distances from the image acquisition device model are within a preset distance range; The area corresponding to the multiple spatial positions is determined to be a virtual identification sub-area corresponding to the image acquisition device model.

[0095] In a possible implementation, the recognition region determination module 403 is specifically configured to determine parameter information of each image acquisition device model based on the virtual recognition sub-regions respectively corresponding to each image acquisition device model; Based on the parameter information of each image acquisition device model, adjusting the parameter information of each image acquisition device in the area to be identified; Based on the adjusted parameter information of each of the image acquisition devices, an object recognition area of ​​each of the image acquisition devices is determined.

[0096] In a possible implementation, the parameter information of the image acquisition device model includes: the focal length of the image acquisition device model and the gimbal angle of the image acquisition device model; the above-mentioned identification area determination module 403 is also specifically used to adjust the focal length and gimbal angle of each image acquisition device in the area to be identified based on the focal length and gimbal angle of each image acquisition device model.

[0097] In a possible implementation, the object recognition module 404 is specifically configured to determine whether the clarity of the target image is greater than or equal to a preset clarity threshold; When it is determined that the clarity of the target image is greater than or equal to the preset clarity threshold, performing object recognition on the area to be recognized based on the target image by using an object recognition model; When it is determined that the clarity of the target image is less than the preset clarity threshold, the pre-deployed radar equipment is used to perform object recognition on the area to be recognized.

[0098] In a possible implementation, the object recognition module 404 is further specifically configured to input the target image into the object recognition model to obtain a recognition result of the target image; If the recognition result is that the target object is recognized, an alarm is issued for the area to be recognized based on a predetermined alarm strategy.

[0099] It can be seen from the technical solution provided by the above embodiments of the present application that, in the embodiments of the present application, first, a digital twin space model of the area to be identified is constructed; the digital twin space model includes multiple image acquisition device models; each image acquisition device model corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; then, the virtual identification area corresponding to the area to be identified in the digital twin space model and the virtual identification sub-areas corresponding to each image acquisition device model in the virtual identification area are determined; then, based on the virtual identification sub-areas corresponding to each image acquisition device model, the object identification area of ​​each image acquisition device is determined; finally, each image acquisition device is controlled to perform image acquisition on its corresponding object identification area to obtain a target image, and based on the target image, object identification is performed on the area to be identified. It can be seen that through the embodiments of the present application, the digital twin space model can be synchronized with the area to be identified in real time, and the object recognition area of ​​each image acquisition device is determined by the virtual recognition sub-areas corresponding to each image acquisition device model in the digital twin space model, and then each image acquisition device is controlled to perform image acquisition on its corresponding object recognition area to obtain the target image, so as to perform object recognition on the area to be identified based on the target image, which enables the image acquisition device in the area to be identified to capture the changes in the image of the object recognition area in a timely and accurate manner, thereby improving the accuracy and efficiency of object recognition.

[0100] It should be noted that the object recognition device provided in the embodiment of the present application and the object recognition method provided in the embodiment of the present application are based on the same inventive concept, so the specific implementation of this embodiment can refer to the implementation of the aforementioned object recognition method, and the repeated parts will not be repeated.

[0101] Corresponding to the object recognition method provided in the above embodiment, based on the same technical concept, the embodiment of the present application also provides an electronic device, which is used to execute the above object recognition method. Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 501 and memory 502, and the memory 502 may store one or more storage applications or data. Among them, the memory 502 may be a temporary storage or a permanent storage. The application stored in the memory 502 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 501 may be configured to communicate with the memory 502 to execute a series of computer executable instructions in the memory 502 on the electronic device. The electronic device may also include one or more power supplies 503, one or more wired or wireless network interfaces 504, one or more input and output interfaces 505, and one or more keyboards 506.

[0102] Specifically in this embodiment, the electronic device includes a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other through the bus; the memory is used to store a computer program; the processor is used to execute the program stored in the memory to implement the following method steps: Constructing a digital twin space model of the area to be identified; the digital twin space model includes a plurality of image acquisition device models; each of the image acquisition device models corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; Determine a virtual recognition area corresponding to the area to be recognized in the digital twin space model, and virtual recognition sub-areas in the virtual recognition area corresponding to each of the image acquisition device models; Determining the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to the image acquisition device models; Each of the image acquisition devices is controlled to acquire images of the object recognition area corresponding to the respective one to obtain a target image, and based on the target image, object recognition is performed on the area to be recognized.

[0103] It can be seen from the technical solution provided by the above embodiments of the present application that, in the embodiments of the present application, first, a digital twin space model of the area to be identified is constructed; the digital twin space model includes multiple image acquisition device models; each image acquisition device model corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; then, the virtual identification area corresponding to the area to be identified in the digital twin space model and the virtual identification sub-areas corresponding to each image acquisition device model in the virtual identification area are determined; then, based on the virtual identification sub-areas corresponding to each image acquisition device model, the object identification area of ​​each image acquisition device is determined; finally, each image acquisition device is controlled to perform image acquisition on its corresponding object identification area to obtain a target image, and based on the target image, object identification is performed on the area to be identified. It can be seen that through the embodiments of the present application, the digital twin space model can be synchronized with the area to be identified in real time, and the object recognition area of ​​each image acquisition device is determined by the virtual recognition sub-areas corresponding to each image acquisition device model in the digital twin space model, and then each image acquisition device is controlled to perform image acquisition on its corresponding object recognition area to obtain the target image, so as to perform object recognition on the area to be identified based on the target image, which enables the image acquisition device in the area to be identified to capture the changes in the image of the object recognition area in a timely and accurate manner, thereby improving the accuracy and efficiency of object recognition.

[0104] The present application also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented: Constructing a digital twin space model of the area to be identified; the digital twin space model includes a plurality of image acquisition device models; each of the image acquisition device models corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; Determine a virtual recognition area corresponding to the area to be recognized in the digital twin space model, and virtual recognition sub-areas in the virtual recognition area corresponding to each of the image acquisition device models; Determining the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to the image acquisition device models; Each of the image acquisition devices is controlled to acquire images of the object recognition area corresponding to the respective one to obtain a target image, and based on the target image, object recognition is performed on the area to be recognized.

[0105] It can be seen from the technical solution provided by the above embodiments of the present application that, in the embodiments of the present application, first, a digital twin space model of the area to be identified is constructed; the digital twin space model includes multiple image acquisition device models; each image acquisition device model corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; then, the virtual identification area corresponding to the area to be identified in the digital twin space model and the virtual identification sub-areas corresponding to each image acquisition device model in the virtual identification area are determined; then, based on the virtual identification sub-areas corresponding to each image acquisition device model, the object identification area of ​​each image acquisition device is determined; finally, each image acquisition device is controlled to perform image acquisition on its corresponding object identification area to obtain a target image, and based on the target image, object identification is performed on the area to be identified. It can be seen that through the embodiments of the present application, the digital twin space model can be synchronized with the area to be identified in real time, and the object recognition area of ​​each image acquisition device is determined by the virtual recognition sub-areas corresponding to each image acquisition device model in the digital twin space model, and then each image acquisition device is controlled to perform image acquisition on its corresponding object recognition area to obtain the target image, so as to perform object recognition on the area to be identified based on the target image, which enables the image acquisition device in the area to be identified to capture the changes in the image of the object recognition area in a timely and accurate manner, thereby improving the accuracy and efficiency of object recognition.

[0106] The embodiment of the present application also provides a computer program product. When the computer program is executed by a processor, each process of the above-mentioned object recognition method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] In a typical configuration, an electronic device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0112] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0113] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0115] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0116] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An object recognition method, characterized in that: include: Construct a digital twin spatial model of the area to be identified; The digital twin space model includes a plurality of image acquisition device models; each of the image acquisition device models corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; Determine a virtual recognition area corresponding to the area to be recognized in the digital twin space model, and virtual recognition sub-areas in the virtual recognition area corresponding to each of the image acquisition device models; Determining the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to the image acquisition device models; Each of the image acquisition devices is controlled to acquire images of the object recognition area corresponding to the respective one to obtain a target image, and based on the target image, object recognition is performed on the area to be recognized.

2. The method according to claim 1, characterized in that The step of constructing a digital twin spatial model of the area to be identified includes: Acquire a two-dimensional image and a three-dimensional image of the area to be identified; Constructing a preset device model of each preset device in the to-be-identified area in the digital twin space model; Based on the two-dimensional image and the three-dimensional image, and the identification information of each of the preset devices, determining the position information of each of the preset device models in the digital twin space model; Based on the two-dimensional image and the three-dimensional image, as well as the position information of each of the preset device models in the digital twin space model, a digital twin space model of the area to be identified is constructed.

3. The method according to claim 2, characterized in that The determining of the virtual recognition sub-areas in the virtual recognition area respectively corresponding to the image acquisition device models comprises: For each of the image acquisition device models, determining a plurality of spatial positions whose distances from the image acquisition device model are within a preset distance range; The area corresponding to the multiple spatial positions is determined to be a virtual identification sub-area corresponding to the image acquisition device model.

4. The method according to claim 3, characterized in that The determining of the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to the image acquisition device models comprises: Determining parameter information of each image acquisition device model based on the virtual identification sub-areas respectively corresponding to each image acquisition device model; Based on the parameter information of each image acquisition device model, adjusting the parameter information of each image acquisition device in the area to be identified; Based on the adjusted parameter information of each of the image acquisition devices, an object recognition area of ​​each of the image acquisition devices is determined.

5. The method according to claim 4, characterized in that The parameter information of the image acquisition device model includes: the focal length of the image acquisition device model and the pan / tilt angle of the image acquisition device model; The step of adjusting the parameter information of each image acquisition device in the to-be-identified area based on the parameter information of each image acquisition device model includes: Based on the focal length and pan / tilt angle of each image acquisition device model, the focal length and pan / tilt angle of each image acquisition device in the area to be identified are adjusted.

6. The method according to claim 1, characterized in that The performing object recognition on the to-be-recognized area based on the target image includes: Determining whether the clarity of the target image is greater than or equal to a preset clarity threshold; When it is determined that the clarity of the target image is greater than or equal to the preset clarity threshold, performing object recognition on the area to be recognized based on the target image by using an object recognition model; When it is determined that the clarity of the target image is less than the preset clarity threshold, the pre-deployed radar equipment is used to perform object recognition on the area to be recognized.

7. The method according to claim 6, characterized in that The performing object recognition on the area to be recognized based on the target image by using the object recognition model includes: Inputting the target image into the object recognition model to obtain a recognition result of the target image; If the recognition result is that the target object is recognized, an alarm is issued for the area to be recognized based on a predetermined alarm strategy.

8. An object recognition device, characterized in that: include: The spatial model building module is used to build a digital twin spatial model of the area to be identified; The digital twin space model includes multiple image acquisition device models; Each of the image acquisition device models corresponds to a plurality of image acquisition devices pre-configured in the area to be identified; A virtual area determination module, used to determine a virtual identification area corresponding to the area to be identified in the digital twin space model, and virtual identification sub-areas in the virtual identification area corresponding to each of the image acquisition device models; A recognition area determination module, used to determine the object recognition area of ​​each image acquisition device based on the virtual recognition sub-areas respectively corresponding to each image acquisition device model; The object recognition module is used to control each of the image acquisition devices to perform image acquisition on the corresponding object recognition area to obtain a target image, and based on the target image, perform object recognition on the area to be recognized.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other through the bus; the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement the object recognition method steps as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the object recognition method according to any one of claims 1 to 7 are implemented.

Images